TMPFS over NFSv4

TMPFS over NFSv4

[Tharindu Rukshan Bamunuarachchi]

dear All,

I tried to export tmpfs file system over NFS and got followin oops ....
this kernel is provided with SLES 11 and tainted due to OFED installation.

I am using NFSv4. Please help me to find the root cause if you feel free ....



BUG: unable to handle kernel NULL pointer dereference at 00000000000000b0
IP: __vm_enough_memory+0xf9/0x14e
PGD 0
Oops: 0000 [1] SMP
last sysfs file:
/sys/devices/pci0000:00/0000:00:09.0/0000:24:00.0/infiniband/mlx4_1/node_desc
CPU 0
Modules linked in: md5 mmfs26(X) mmfslinux(X) tracedev(X) nfsd lockd
nfs_acl auth_rpcgss sunrpc exportfs bonding binfmt_misc microcode
rdma_ucm(N) ib_sdp(N) rdma_cm(N) iw_cm(N) ib_addr(N) ib_ipoib(N)
ib_cm(N) ib_sa(N) ipv6 ib_uverbs(N) ib_umad(N) mlx4_en(N) mlx4_ib(N)
ib_mthca(N) ib_mad(N) ib_core(N) fuse ext2 xfs loop dm_mod cdc_ether
usbnet i2c_i801 rtc_cmos shpchp button joydev mlx4_core(N) rtc_core
pcspkr pci_hotplug sr_mod rtc_lib mii bnx2 i2c_core cdrom sg usbhid
hid ff_memless uhci_hcd ehci_hcd sd_mod crc_t10dif usbcore edd ext3
mbcache jbd fan thermal processor thermal_sys hwmon ide_pci_generic
ide_core ata_generic ata_piix libata dock megaraid_sas scsi_mod [last
unloaded: tracedev]
Supported: No, Unsupported modules are loaded
Pid: 8855, comm: nfsd Tainted: G 2.6.27.45-0.1-default #1
RIP: 0010: __vm_enough_memory+0xf9/0x14e
RSP: 0018:ffff8803642cd780 EFLAGS: 00010202
RAX: 00000000002f151c RBX: 0000000012643f22 RCX: 0000000000400293
RDX: 0000000000000032 RSI: 0000000000000001 RDI: 00000000002f151c
RBP: 0000000000000001 R08: 00000000ffffffe5 R09: 0000000000000000
R10: ffffffff806eb5c8 R11: ffffffff80317108 R12: 0000000000000001
R13: 0000000000000000 R14: 0000000000001000 R15: ffff88037b93b738
FS: 0000000000000000(0000) GS:ffffffff80a43080(0000) knlGS:0000000000000000
CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00000000000000b0 CR3: 0000000000201000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process nfsd (pid: 8855, threadinfo ffff8803642cc000, task ffff88036f140380)
Stack: ffff88037b93b668 ffff88037009de40 ffff88037b93b668 0000000000000000
ffff88037b93b601 ffffffff802a8573 ffffffff80a33680 ffffffff80a30730
0000000000000000 0000000300000002 ffff8803642cd930 0000000000000000
Call Trace:
shmem_getpage+0x4d8/0x764
generic_perform_write+0xae/0x1b5
generic_file_buffered_write+0x80/0x130
__generic_file_aio_write_nolock+0x349/0x37d
generic_file_aio_write+0x64/0xc4
do_sync_readv_writev+0xc0/0x107
do_readv_writev+0xb2/0x18b
nfsd_vfs_write+0x10a/0x328 [nfsd]
nfsd_write+0x79/0xe2 [nfsd]
nfsd4_write+0xd9/0x10d [nfsd]
nfsd4_proc_compound+0x1bd/0x2c7 [nfsd]
nfsd_dispatch+0xdd/0x1b9 [nfsd]
svc_process+0x3d8/0x700 [sunrpc]
nfsd+0x1b1/0x27e [nfsd]
kthread+0x47/0x73
child_rip+0xa/0x11


Code: 00 48 29 c3 48 63 05 49 1a 45 00 48 0f af d8 48 89 d8 48 f7 f1
45 85 e4 48 89 c7 75 07 48 c1 e8 05 48 29 c7 48 8b 0d b9 2f 86 00 <49>
8b b5 b0 00 00 00 48 8b 15 43 2f 86 00 b8 01 00 00 00 48 85
RIP __vm_enough_memory+0xf9/0x14e
RSP <ffff8803642cd780>
CR2: 00000000000000b0
---[ end trace ca3d7c15970cb4b6 ]---

__
tharindu.info

"those that can, do. Those that can’t, complain." -- Linus

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: TMPFS over NFSv4

[Hugh Dickins]

On Fri, 21 May 2010, Tharindu Rukshan Bamunuarachchi wrote:
> 
> I tried to export tmpfs file system over NFS and got followin oops ....
> this kernel is provided with SLES 11 and tainted due to OFED installation.
> 
> I am using NFSv4. Please help me to find the root cause if you feel free ....
> 
> 
> BUG: unable to handle kernel NULL pointer dereference at 00000000000000b0
> IP: __vm_enough_memory+0xf9/0x14e
> PGD 0
> Oops: 0000 [1] SMP
> last sysfs file:
> /sys/devices/pci0000:00/0000:00:09.0/0000:24:00.0/infiniband/mlx4_1/node_desc
> CPU 0
> Modules linked in: blah blah blah
> Supported: No, Unsupported modules are loaded
> Pid: 8855, comm: nfsd Tainted: G 2.6.27.45-0.1-default #1
> RIP: 0010: __vm_enough_memory+0xf9/0x14e
...
> Process nfsd (pid: 8855, threadinfo ffff8803642cc000, task ffff88036f140380)
> Stack: ffff88037b93b668 ffff88037009de40 ffff88037b93b668 0000000000000000
> ffff88037b93b601 ffffffff802a8573 ffffffff80a33680 ffffffff80a30730
> 0000000000000000 0000000300000002 ffff8803642cd930 0000000000000000
> Call Trace:
> shmem_getpage+0x4d8/0x764
> generic_perform_write+0xae/0x1b5
> generic_file_buffered_write+0x80/0x130
> __generic_file_aio_write_nolock+0x349/0x37d
> generic_file_aio_write+0x64/0xc4
> do_sync_readv_writev+0xc0/0x107
> do_readv_writev+0xb2/0x18b
> nfsd_vfs_write+0x10a/0x328 [nfsd]
> nfsd_write+0x79/0xe2 [nfsd]
> nfsd4_write+0xd9/0x10d [nfsd]
> nfsd4_proc_compound+0x1bd/0x2c7 [nfsd]
> nfsd_dispatch+0xdd/0x1b9 [nfsd]
> svc_process+0x3d8/0x700 [sunrpc]
> nfsd+0x1b1/0x27e [nfsd]
> kthread+0x47/0x73
> child_rip+0xa/0x11

I believe that was fixed in 2.6.28 by the patch below:
please would you try it, and if it works for you, then
I'll ask for it to be included in the next 2.6.27-stable,
which I expect SLES 11 will include in an update later.
Strange that more people haven't suffered from it...

Hugh

commit 731572d39fcd3498702eda4600db4c43d51e0b26
Author: Alan Cox <alan@redhat.com>
Date:   Wed Oct 29 14:01:20 2008 -0700

    nfsd: fix vm overcommit crash
    
    Junjiro R.  Okajima reported a problem where knfsd crashes if you are
    using it to export shmemfs objects and run strict overcommit.  In this
    situation the current->mm based modifier to the overcommit goes through a
    NULL pointer.
    
    We could simply check for NULL and skip the modifier but we've caught
    other real bugs in the past from mm being NULL here - cases where we did
    need a valid mm set up (eg the exec bug about a year ago).
    
    To preserve the checks and get the logic we want shuffle the checking
    around and add a new helper to the vm_ security wrappers
    
    Also fix a current->mm reference in nommu that should use the passed mm
    
    [akpm@linux-foundation.org: coding-style fixes]
    [akpm@linux-foundation.org: fix build]
    Reported-by: Junjiro R. Okajima <hooanon05@yahoo.co.jp>
    Acked-by: James Morris <jmorris@namei.org>
    Signed-off-by: Alan Cox <alan@redhat.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

diff --git a/include/linux/security.h b/include/linux/security.h
index f5c4a51..c13f1ce 100644
--- a/include/linux/security.h
+++ b/include/linux/security.h
@@ -1585,6 +1585,7 @@ int security_syslog(int type);
 int security_settime(struct timespec *ts, struct timezone *tz);
 int security_vm_enough_memory(long pages);
 int security_vm_enough_memory_mm(struct mm_struct *mm, long pages);
+int security_vm_enough_memory_kern(long pages);
 int security_bprm_alloc(struct linux_binprm *bprm);
 void security_bprm_free(struct linux_binprm *bprm);
 void security_bprm_apply_creds(struct linux_binprm *bprm, int unsafe);
@@ -1820,6 +1821,11 @@ static inline int security_vm_enough_memory(long pages)
 	return cap_vm_enough_memory(current->mm, pages);
 }
 
+static inline int security_vm_enough_memory_kern(long pages)
+{
+	return cap_vm_enough_memory(current->mm, pages);
+}
+
 static inline int security_vm_enough_memory_mm(struct mm_struct *mm, long pages)
 {
 	return cap_vm_enough_memory(mm, pages);
diff --git a/mm/mmap.c b/mm/mmap.c
index 74f4d15..de14ac2 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -175,7 +175,8 @@ int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
 
 	/* Don't let a single process grow too big:
 	   leave 3% of the size of this process for other processes */
-	allowed -= mm->total_vm / 32;
+	if (mm)
+		allowed -= mm->total_vm / 32;
 
 	/*
 	 * cast `allowed' as a signed long because vm_committed_space
diff --git a/mm/nommu.c b/mm/nommu.c
index 2696b24..7695dc8 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -1454,7 +1454,8 @@ int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
 
 	/* Don't let a single process grow too big:
 	   leave 3% of the size of this process for other processes */
-	allowed -= current->mm->total_vm / 32;
+	if (mm)
+		allowed -= mm->total_vm / 32;
 
 	/*
 	 * cast `allowed' as a signed long because vm_committed_space
diff --git a/mm/shmem.c b/mm/shmem.c
index d38d7e6..0ed0752 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -161,8 +161,8 @@ static inline struct shmem_sb_info *SHMEM_SB(struct super_block *sb)
  */
 static inline int shmem_acct_size(unsigned long flags, loff_t size)
 {
-	return (flags & VM_ACCOUNT)?
-		security_vm_enough_memory(VM_ACCT(size)): 0;
+	return (flags & VM_ACCOUNT) ?
+		security_vm_enough_memory_kern(VM_ACCT(size)) : 0;
 }
 
 static inline void shmem_unacct_size(unsigned long flags, loff_t size)
@@ -179,8 +179,8 @@ static inline void shmem_unacct_size(unsigned long flags, loff_t size)
  */
 static inline int shmem_acct_block(unsigned long flags)
 {
-	return (flags & VM_ACCOUNT)?
-		0: security_vm_enough_memory(VM_ACCT(PAGE_CACHE_SIZE));
+	return (flags & VM_ACCOUNT) ?
+		0 : security_vm_enough_memory_kern(VM_ACCT(PAGE_CACHE_SIZE));
 }
 
 static inline void shmem_unacct_blocks(unsigned long flags, long pages)
diff --git a/security/security.c b/security/security.c
index 255b085..c0acfa7 100644
--- a/security/security.c
+++ b/security/security.c
@@ -198,14 +198,23 @@ int security_settime(struct timespec *ts, struct timezone *tz)
 
 int security_vm_enough_memory(long pages)
 {
+	WARN_ON(current->mm == NULL);
 	return security_ops->vm_enough_memory(current->mm, pages);
 }
 
 int security_vm_enough_memory_mm(struct mm_struct *mm, long pages)
 {
+	WARN_ON(mm == NULL);
 	return security_ops->vm_enough_memory(mm, pages);
 }
 
+int security_vm_enough_memory_kern(long pages)
+{
+	/* If current->mm is a kernel thread then we will pass NULL,
+	   for this specific case that is fine */
+	return security_ops->vm_enough_memory(current->mm, pages);
+}
+
 int security_bprm_alloc(struct linux_binprm *bprm)
 {
 	return security_ops->bprm_alloc_security(bprm);

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: TMPFS over NFSv4

[Tharindu Rukshan Bamunuarachchi]

thankx a lot Hugh ... I will try this out ... (bit harder patch
already patched SLES kernel :-p ) ....

BTW, what does Alan means by "strict overcommit" ?

e.g.
i did not see this issues with "0 > /proc/sys/vm/overcommit_accounting"
But this happened several times with "2 > /proc/sys/vm/overcommit_accounting"

any clue ?

we are suffering everyday ..... :-|

__
tharindu.info

"those that can, do. Those that can’t, complain." -- Linus



On Fri, May 21, 2010 at 9:55 PM, Hugh Dickins <hughd@google.com> wrote:
> On Fri, 21 May 2010, Tharindu Rukshan Bamunuarachchi wrote:
>>
>> I tried to export tmpfs file system over NFS and got followin oops ....
>> this kernel is provided with SLES 11 and tainted due to OFED installation.
>>
>> I am using NFSv4. Please help me to find the root cause if you feel free ....
>>
>>
>> BUG: unable to handle kernel NULL pointer dereference at 00000000000000b0
>> IP: __vm_enough_memory+0xf9/0x14e
>> PGD 0
>> Oops: 0000 [1] SMP
>> last sysfs file:
>> /sys/devices/pci0000:00/0000:00:09.0/0000:24:00.0/infiniband/mlx4_1/node_desc
>> CPU 0
>> Modules linked in: blah blah blah
>> Supported: No, Unsupported modules are loaded
>> Pid: 8855, comm: nfsd Tainted: G 2.6.27.45-0.1-default #1
>> RIP: 0010: __vm_enough_memory+0xf9/0x14e
> ...
>> Process nfsd (pid: 8855, threadinfo ffff8803642cc000, task ffff88036f140380)
>> Stack: ffff88037b93b668 ffff88037009de40 ffff88037b93b668 0000000000000000
>> ffff88037b93b601 ffffffff802a8573 ffffffff80a33680 ffffffff80a30730
>> 0000000000000000 0000000300000002 ffff8803642cd930 0000000000000000
>> Call Trace:
>> shmem_getpage+0x4d8/0x764
>> generic_perform_write+0xae/0x1b5
>> generic_file_buffered_write+0x80/0x130
>> __generic_file_aio_write_nolock+0x349/0x37d
>> generic_file_aio_write+0x64/0xc4
>> do_sync_readv_writev+0xc0/0x107
>> do_readv_writev+0xb2/0x18b
>> nfsd_vfs_write+0x10a/0x328 [nfsd]
>> nfsd_write+0x79/0xe2 [nfsd]
>> nfsd4_write+0xd9/0x10d [nfsd]
>> nfsd4_proc_compound+0x1bd/0x2c7 [nfsd]
>> nfsd_dispatch+0xdd/0x1b9 [nfsd]
>> svc_process+0x3d8/0x700 [sunrpc]
>> nfsd+0x1b1/0x27e [nfsd]
>> kthread+0x47/0x73
>> child_rip+0xa/0x11
>
> I believe that was fixed in 2.6.28 by the patch below:
> please would you try it, and if it works for you, then
> I'll ask for it to be included in the next 2.6.27-stable,
> which I expect SLES 11 will include in an update later.
> Strange that more people haven't suffered from it...
>
> Hugh
>
> commit 731572d39fcd3498702eda4600db4c43d51e0b26
> Author: Alan Cox <alan@redhat.com>
> Date:   Wed Oct 29 14:01:20 2008 -0700
>
>    nfsd: fix vm overcommit crash
>
>    Junjiro R.  Okajima reported a problem where knfsd crashes if you are
>    using it to export shmemfs objects and run strict overcommit.  In this
>    situation the current->mm based modifier to the overcommit goes through a
>    NULL pointer.
>
>    We could simply check for NULL and skip the modifier but we've caught
>    other real bugs in the past from mm being NULL here - cases where we did
>    need a valid mm set up (eg the exec bug about a year ago).
>
>    To preserve the checks and get the logic we want shuffle the checking
>    around and add a new helper to the vm_ security wrappers
>
>    Also fix a current->mm reference in nommu that should use the passed mm
>
>    [akpm@linux-foundation.org: coding-style fixes]
>    [akpm@linux-foundation.org: fix build]
>    Reported-by: Junjiro R. Okajima <hooanon05@yahoo.co.jp>
>    Acked-by: James Morris <jmorris@namei.org>
>    Signed-off-by: Alan Cox <alan@redhat.com>
>    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
>    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
>
> diff --git a/include/linux/security.h b/include/linux/security.h
> index f5c4a51..c13f1ce 100644
> --- a/include/linux/security.h
> +++ b/include/linux/security.h
> @@ -1585,6 +1585,7 @@ int security_syslog(int type);
>  int security_settime(struct timespec *ts, struct timezone *tz);
>  int security_vm_enough_memory(long pages);
>  int security_vm_enough_memory_mm(struct mm_struct *mm, long pages);
> +int security_vm_enough_memory_kern(long pages);
>  int security_bprm_alloc(struct linux_binprm *bprm);
>  void security_bprm_free(struct linux_binprm *bprm);
>  void security_bprm_apply_creds(struct linux_binprm *bprm, int unsafe);
> @@ -1820,6 +1821,11 @@ static inline int security_vm_enough_memory(long pages)
>        return cap_vm_enough_memory(current->mm, pages);
>  }
>
> +static inline int security_vm_enough_memory_kern(long pages)
> +{
> +       return cap_vm_enough_memory(current->mm, pages);
> +}
> +
>  static inline int security_vm_enough_memory_mm(struct mm_struct *mm, long pages)
>  {
>        return cap_vm_enough_memory(mm, pages);
> diff --git a/mm/mmap.c b/mm/mmap.c
> index 74f4d15..de14ac2 100644
> --- a/mm/mmap.c
> +++ b/mm/mmap.c
> @@ -175,7 +175,8 @@ int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
>
>        /* Don't let a single process grow too big:
>           leave 3% of the size of this process for other processes */
> -       allowed -= mm->total_vm / 32;
> +       if (mm)
> +               allowed -= mm->total_vm / 32;
>
>        /*
>         * cast `allowed' as a signed long because vm_committed_space
> diff --git a/mm/nommu.c b/mm/nommu.c
> index 2696b24..7695dc8 100644
> --- a/mm/nommu.c
> +++ b/mm/nommu.c
> @@ -1454,7 +1454,8 @@ int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
>
>        /* Don't let a single process grow too big:
>           leave 3% of the size of this process for other processes */
> -       allowed -= current->mm->total_vm / 32;
> +       if (mm)
> +               allowed -= mm->total_vm / 32;
>
>        /*
>         * cast `allowed' as a signed long because vm_committed_space
> diff --git a/mm/shmem.c b/mm/shmem.c
> index d38d7e6..0ed0752 100644
> --- a/mm/shmem.c
> +++ b/mm/shmem.c
> @@ -161,8 +161,8 @@ static inline struct shmem_sb_info *SHMEM_SB(struct super_block *sb)
>  */
>  static inline int shmem_acct_size(unsigned long flags, loff_t size)
>  {
> -       return (flags & VM_ACCOUNT)?
> -               security_vm_enough_memory(VM_ACCT(size)): 0;
> +       return (flags & VM_ACCOUNT) ?
> +               security_vm_enough_memory_kern(VM_ACCT(size)) : 0;
>  }
>
>  static inline void shmem_unacct_size(unsigned long flags, loff_t size)
> @@ -179,8 +179,8 @@ static inline void shmem_unacct_size(unsigned long flags, loff_t size)
>  */
>  static inline int shmem_acct_block(unsigned long flags)
>  {
> -       return (flags & VM_ACCOUNT)?
> -               0: security_vm_enough_memory(VM_ACCT(PAGE_CACHE_SIZE));
> +       return (flags & VM_ACCOUNT) ?
> +               0 : security_vm_enough_memory_kern(VM_ACCT(PAGE_CACHE_SIZE));
>  }
>
>  static inline void shmem_unacct_blocks(unsigned long flags, long pages)
> diff --git a/security/security.c b/security/security.c
> index 255b085..c0acfa7 100644
> --- a/security/security.c
> +++ b/security/security.c
> @@ -198,14 +198,23 @@ int security_settime(struct timespec *ts, struct timezone *tz)
>
>  int security_vm_enough_memory(long pages)
>  {
> +       WARN_ON(current->mm == NULL);
>        return security_ops->vm_enough_memory(current->mm, pages);
>  }
>
>  int security_vm_enough_memory_mm(struct mm_struct *mm, long pages)
>  {
> +       WARN_ON(mm == NULL);
>        return security_ops->vm_enough_memory(mm, pages);
>  }
>
> +int security_vm_enough_memory_kern(long pages)
> +{
> +       /* If current->mm is a kernel thread then we will pass NULL,
> +          for this specific case that is fine */
> +       return security_ops->vm_enough_memory(current->mm, pages);
> +}
> +
>  int security_bprm_alloc(struct linux_binprm *bprm)
>  {
>        return security_ops->bprm_alloc_security(bprm);
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: TMPFS over NFSv4

[Hugh Dickins]

On Mon, May 24, 2010 at 2:26 AM, Tharindu Rukshan Bamunuarachchi
<btharindu@gmail.com> wrote:
> thankx a lot Hugh ... I will try this out ... (bit harder patch
> already patched SLES kernel :-p ) ....

If patch conflicts are a problem, you really only need to put in the
two-liner patch to mm/mmap.c: Alan was seeking perfection in
the rest of the patch, but you can get away without it.

>
> BTW, what does Alan means by "strict overcommit" ?

Ah, that phrase, yes, it's a nonsense, but many of us do say it by mistake.
Alan meant to say "strict no-overcommit".

>
> e.g.
> i did not see this issues with "0 > /proc/sys/vm/overcommit_accounting"

I assume "overcommit_accounting" is either a typo for "overcommit_memory",
or SLES gives "overcommit_memory" a slightly different name.

0 means overcommit memory (let people allocate more private writable user
memory than there is actually ram+swap to back), but throw in a check against
really wild allocation requests.  1 omits even that check.

> But this happened several times with "2 > /proc/sys/vm/overcommit_accounting"

2 means account for all private writable memory and fail any allocation which
would take the system over the edge - the edge being defined roughly by
overcommit_ratio * (ram+swap)  (I expect there's a divisor needed in there!)
i.e. 2 means strict no-overcommit.

So what you see fits with what Alan was fixing.

Hugh

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: TMPFS over NFSv4

[Alan Cox]

On Mon, 24 May 2010 10:26:39 +0100
Tharindu Rukshan Bamunuarachchi <btharindu@gmail.com> wrote:

> thankx a lot Hugh ... I will try this out ... (bit harder patch
> already patched SLES kernel :-p ) ....
> 
> BTW, what does Alan means by "strict overcommit" ?

Strict overcommit works like banks should. It tries to ensure that at any
point it has sufficient swap and memory to fulfill any possible use of
allocated address space. So in strict overcommit mode you should almost
never see an OOM kill (there are perverse cases as always), but you will
need a lot more swap that may well never be used.

In the normal mode the kernel works like the US banking system and makes
speculative guesses that all the resources it hands out will never be
needed at once. That has the corresponding risk that one day it might at
which point you get a meltdown (or in the kernel case OOM kills)

Alan

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: TMPFS over NFSv4

[Alan Cox]

On Mon, 24 May 2010 02:57:30 -0700
Hugh Dickins <hughd@google.com> wrote:

> On Mon, May 24, 2010 at 2:26 AM, Tharindu Rukshan Bamunuarachchi
> <btharindu@gmail.com> wrote:
> > thankx a lot Hugh ... I will try this out ... (bit harder patch
> > already patched SLES kernel :-p ) ....
> 
> If patch conflicts are a problem, you really only need to put in the
> two-liner patch to mm/mmap.c: Alan was seeking perfection in
> the rest of the patch, but you can get away without it.
> 
> >
> > BTW, what does Alan means by "strict overcommit" ?
> 
> Ah, that phrase, yes, it's a nonsense, but many of us do say it by mistake.
> Alan meant to say "strict no-overcommit".

No I always meant to say 'strict overcommit'. It avoids excess negatives
and "no noovercommit" discussions.

I guess 'strict overcommit control' would have been clearer 8)

Alan

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: TMPFS over NFSv4

[Tharindu Rukshan Bamunuarachchi]

Got it cleared.

BTW, nice example ... US Banking System :-)

__
tharindu.info

"those that can, do. Those that can’t, complain." -- Linus



On Mon, May 24, 2010 at 11:02 AM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> On Mon, 24 May 2010 10:26:39 +0100
> Tharindu Rukshan Bamunuarachchi <btharindu@gmail.com> wrote:
>
>> thankx a lot Hugh ... I will try this out ... (bit harder patch
>> already patched SLES kernel :-p ) ....
>>
>> BTW, what does Alan means by "strict overcommit" ?
>
> Strict overcommit works like banks should. It tries to ensure that at any
> point it has sufficient swap and memory to fulfill any possible use of
> allocated address space. So in strict overcommit mode you should almost
> never see an OOM kill (there are perverse cases as always), but you will
> need a lot more swap that may well never be used.
>
> In the normal mode the kernel works like the US banking system and makes
> speculative guesses that all the resources it hands out will never be
> needed at once. That has the corresponding risk that one day it might at
> which point you get a meltdown (or in the kernel case OOM kills)
>
> Alan
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: TMPFS over NFSv4

[Tharindu Rukshan Bamunuarachchi]

hi Hugh.

On Mon, May 24, 2010 at 10:57 AM, Hugh Dickins <hughd@google.com> wrote:
> On Mon, May 24, 2010 at 2:26 AM, Tharindu Rukshan Bamunuarachchi
>
> If patch conflicts are a problem, you really only need to put in the
> two-liner patch to mm/mmap.c: Alan was seeking perfection in
> the rest of the patch, but you can get away without it.
>

I will just add it by hand.

>
>
> So what you see fits with what Alan was fixing.
>

Yes, of courese.

BTW, that was typo and it should be overcommit_memory.
I have done testing in our test severs and "2 > overcommit_memory"
triggers the issue.

OTOH, I have reported the issue Novell.
Hope they will apply necessary patch in next release.


> Hugh
>


Thankx a lot/

__
tharindu

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: TMPFS over NFSv4

[Hugh Dickins]

Hi Greg,

On Mon, 24 May 2010, Alan Cox wrote:
> On Mon, 24 May 2010 02:57:30 -0700
> Hugh Dickins <hughd@google.com> wrote:
> > On Mon, May 24, 2010 at 2:26 AM, Tharindu Rukshan Bamunuarachchi
> > <btharindu@gmail.com> wrote:
> > > thankx a lot Hugh ... I will try this out ... (bit harder patch
> > > already patched SLES kernel :-p ) ....
> > 
> > If patch conflicts are a problem, you really only need to put in the
> > two-liner patch to mm/mmap.c: Alan was seeking perfection in
> > the rest of the patch, but you can get away without it.
> > 
> > >
> > > BTW, what does Alan means by "strict overcommit" ?
> > 
> > Ah, that phrase, yes, it's a nonsense, but many of us do say it by mistake.
> > Alan meant to say "strict no-overcommit".
> 
> No I always meant to say 'strict overcommit'. It avoids excess negatives
> and "no noovercommit" discussions.
> 
> I guess 'strict overcommit control' would have been clearer 8)
> 
> Alan

I see we've just missed 2.6.27.47-rc1, but if there's to be an -rc2,
please include Alan's 2.6.28 oops fix below: which Tharindu appears
to be needing - just now discussed on linux-mm and linux-nfs.
Failing that, please queue it up for 2.6.27.48.

Or if you'd prefer a smaller patch for -stable, then just the mm/mmap.c
part of it should suffice: I think it's fair to say that the rest of the
patch was more precautionary - as Alan describes, for catching other bugs,
so good for an ongoing development tree, but not necessarily in -stable.
(However, Alan may disagree - I've already misrepresented him once here!)

Thanks,
Hugh

commit 731572d39fcd3498702eda4600db4c43d51e0b26
Author: Alan Cox <alan@redhat.com>
Date:   Wed Oct 29 14:01:20 2008 -0700

    nfsd: fix vm overcommit crash
    
    Junjiro R.  Okajima reported a problem where knfsd crashes if you are
    using it to export shmemfs objects and run strict overcommit.  In this
    situation the current->mm based modifier to the overcommit goes through a
    NULL pointer.
    
    We could simply check for NULL and skip the modifier but we've caught
    other real bugs in the past from mm being NULL here - cases where we did
    need a valid mm set up (eg the exec bug about a year ago).
    
    To preserve the checks and get the logic we want shuffle the checking
    around and add a new helper to the vm_ security wrappers
    
    Also fix a current->mm reference in nommu that should use the passed mm
    
    [akpm@linux-foundation.org: coding-style fixes]
    [akpm@linux-foundation.org: fix build]
    Reported-by: Junjiro R. Okajima <hooanon05@yahoo.co.jp>
    Acked-by: James Morris <jmorris@namei.org>
    Signed-off-by: Alan Cox <alan@redhat.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

diff --git a/include/linux/security.h b/include/linux/security.h
index f5c4a51..c13f1ce 100644
--- a/include/linux/security.h
+++ b/include/linux/security.h
@@ -1585,6 +1585,7 @@ int security_syslog(int type);
 int security_settime(struct timespec *ts, struct timezone *tz);
 int security_vm_enough_memory(long pages);
 int security_vm_enough_memory_mm(struct mm_struct *mm, long pages);
+int security_vm_enough_memory_kern(long pages);
 int security_bprm_alloc(struct linux_binprm *bprm);
 void security_bprm_free(struct linux_binprm *bprm);
 void security_bprm_apply_creds(struct linux_binprm *bprm, int unsafe);
@@ -1820,6 +1821,11 @@ static inline int security_vm_enough_memory(long pages)
 	return cap_vm_enough_memory(current->mm, pages);
 }
 
+static inline int security_vm_enough_memory_kern(long pages)
+{
+	return cap_vm_enough_memory(current->mm, pages);
+}
+
 static inline int security_vm_enough_memory_mm(struct mm_struct *mm, long pages)
 {
 	return cap_vm_enough_memory(mm, pages);
diff --git a/mm/mmap.c b/mm/mmap.c
index 74f4d15..de14ac2 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -175,7 +175,8 @@ int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
 
 	/* Don't let a single process grow too big:
 	   leave 3% of the size of this process for other processes */
-	allowed -= mm->total_vm / 32;
+	if (mm)
+		allowed -= mm->total_vm / 32;
 
 	/*
 	 * cast `allowed' as a signed long because vm_committed_space
diff --git a/mm/nommu.c b/mm/nommu.c
index 2696b24..7695dc8 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -1454,7 +1454,8 @@ int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
 
 	/* Don't let a single process grow too big:
 	   leave 3% of the size of this process for other processes */
-	allowed -= current->mm->total_vm / 32;
+	if (mm)
+		allowed -= mm->total_vm / 32;
 
 	/*
 	 * cast `allowed' as a signed long because vm_committed_space
diff --git a/mm/shmem.c b/mm/shmem.c
index d38d7e6..0ed0752 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -161,8 +161,8 @@ static inline struct shmem_sb_info *SHMEM_SB(struct super_block *sb)
  */
 static inline int shmem_acct_size(unsigned long flags, loff_t size)
 {
-	return (flags & VM_ACCOUNT)?
-		security_vm_enough_memory(VM_ACCT(size)): 0;
+	return (flags & VM_ACCOUNT) ?
+		security_vm_enough_memory_kern(VM_ACCT(size)) : 0;
 }
 
 static inline void shmem_unacct_size(unsigned long flags, loff_t size)
@@ -179,8 +179,8 @@ static inline void shmem_unacct_size(unsigned long flags, loff_t size)
  */
 static inline int shmem_acct_block(unsigned long flags)
 {
-	return (flags & VM_ACCOUNT)?
-		0: security_vm_enough_memory(VM_ACCT(PAGE_CACHE_SIZE));
+	return (flags & VM_ACCOUNT) ?
+		0 : security_vm_enough_memory_kern(VM_ACCT(PAGE_CACHE_SIZE));
 }
 
 static inline void shmem_unacct_blocks(unsigned long flags, long pages)
diff --git a/security/security.c b/security/security.c
index 255b085..c0acfa7 100644
--- a/security/security.c
+++ b/security/security.c
@@ -198,14 +198,23 @@ int security_settime(struct timespec *ts, struct timezone *tz)
 
 int security_vm_enough_memory(long pages)
 {
+	WARN_ON(current->mm == NULL);
 	return security_ops->vm_enough_memory(current->mm, pages);
 }
 
 int security_vm_enough_memory_mm(struct mm_struct *mm, long pages)
 {
+	WARN_ON(mm == NULL);
 	return security_ops->vm_enough_memory(mm, pages);
 }
 
+int security_vm_enough_memory_kern(long pages)
+{
+	/* If current->mm is a kernel thread then we will pass NULL,
+	   for this specific case that is fine */
+	return security_ops->vm_enough_memory(current->mm, pages);
+}
+
 int security_bprm_alloc(struct linux_binprm *bprm)
 {
 	return security_ops->bprm_alloc_security(bprm);

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: TMPFS over NFSv4

[Tharindu Rukshan Bamunuarachchi]

hope that the 2.6.27.48 or later will be shipped with SP1 :-)

__
tharindu.info

"those that can, do. Those that can’t, complain." -- Linus



On Tue, May 25, 2010 at 12:46 AM, Hugh Dickins <hughd@google.com> wrote:
> Hi Greg,
>
> On Mon, 24 May 2010, Alan Cox wrote:
>> On Mon, 24 May 2010 02:57:30 -0700
>> Hugh Dickins <hughd@google.com> wrote:
>> > On Mon, May 24, 2010 at 2:26 AM, Tharindu Rukshan Bamunuarachchi
>> > <btharindu@gmail.com> wrote:
>> > > thankx a lot Hugh ... I will try this out ... (bit harder patch
>> > > already patched SLES kernel :-p ) ....
>> >
>> > If patch conflicts are a problem, you really only need to put in the
>> > two-liner patch to mm/mmap.c: Alan was seeking perfection in
>> > the rest of the patch, but you can get away without it.
>> >
>> > >
>> > > BTW, what does Alan means by "strict overcommit" ?
>> >
>> > Ah, that phrase, yes, it's a nonsense, but many of us do say it by mistake.
>> > Alan meant to say "strict no-overcommit".
>>
>> No I always meant to say 'strict overcommit'. It avoids excess negatives
>> and "no noovercommit" discussions.
>>
>> I guess 'strict overcommit control' would have been clearer 8)
>>
>> Alan
>
> I see we've just missed 2.6.27.47-rc1, but if there's to be an -rc2,
> please include Alan's 2.6.28 oops fix below: which Tharindu appears
> to be needing - just now discussed on linux-mm and linux-nfs.
> Failing that, please queue it up for 2.6.27.48.
>
> Or if you'd prefer a smaller patch for -stable, then just the mm/mmap.c
> part of it should suffice: I think it's fair to say that the rest of the
> patch was more precautionary - as Alan describes, for catching other bugs,
> so good for an ongoing development tree, but not necessarily in -stable.
> (However, Alan may disagree - I've already misrepresented him once here!)
>
> Thanks,
> Hugh
>
> commit 731572d39fcd3498702eda4600db4c43d51e0b26
> Author: Alan Cox <alan@redhat.com>
> Date:   Wed Oct 29 14:01:20 2008 -0700
>
>    nfsd: fix vm overcommit crash
>
>    Junjiro R.  Okajima reported a problem where knfsd crashes if you are
>    using it to export shmemfs objects and run strict overcommit.  In this
>    situation the current->mm based modifier to the overcommit goes through a
>    NULL pointer.
>
>    We could simply check for NULL and skip the modifier but we've caught
>    other real bugs in the past from mm being NULL here - cases where we did
>    need a valid mm set up (eg the exec bug about a year ago).
>
>    To preserve the checks and get the logic we want shuffle the checking
>    around and add a new helper to the vm_ security wrappers
>
>    Also fix a current->mm reference in nommu that should use the passed mm
>
>    [akpm@linux-foundation.org: coding-style fixes]
>    [akpm@linux-foundation.org: fix build]
>    Reported-by: Junjiro R. Okajima <hooanon05@yahoo.co.jp>
>    Acked-by: James Morris <jmorris@namei.org>
>    Signed-off-by: Alan Cox <alan@redhat.com>
>    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
>    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
>
> diff --git a/include/linux/security.h b/include/linux/security.h
> index f5c4a51..c13f1ce 100644
> --- a/include/linux/security.h
> +++ b/include/linux/security.h
> @@ -1585,6 +1585,7 @@ int security_syslog(int type);
>  int security_settime(struct timespec *ts, struct timezone *tz);
>  int security_vm_enough_memory(long pages);
>  int security_vm_enough_memory_mm(struct mm_struct *mm, long pages);
> +int security_vm_enough_memory_kern(long pages);
>  int security_bprm_alloc(struct linux_binprm *bprm);
>  void security_bprm_free(struct linux_binprm *bprm);
>  void security_bprm_apply_creds(struct linux_binprm *bprm, int unsafe);
> @@ -1820,6 +1821,11 @@ static inline int security_vm_enough_memory(long pages)
>        return cap_vm_enough_memory(current->mm, pages);
>  }
>
> +static inline int security_vm_enough_memory_kern(long pages)
> +{
> +       return cap_vm_enough_memory(current->mm, pages);
> +}
> +
>  static inline int security_vm_enough_memory_mm(struct mm_struct *mm, long pages)
>  {
>        return cap_vm_enough_memory(mm, pages);
> diff --git a/mm/mmap.c b/mm/mmap.c
> index 74f4d15..de14ac2 100644
> --- a/mm/mmap.c
> +++ b/mm/mmap.c
> @@ -175,7 +175,8 @@ int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
>
>        /* Don't let a single process grow too big:
>           leave 3% of the size of this process for other processes */
> -       allowed -= mm->total_vm / 32;
> +       if (mm)
> +               allowed -= mm->total_vm / 32;
>
>        /*
>         * cast `allowed' as a signed long because vm_committed_space
> diff --git a/mm/nommu.c b/mm/nommu.c
> index 2696b24..7695dc8 100644
> --- a/mm/nommu.c
> +++ b/mm/nommu.c
> @@ -1454,7 +1454,8 @@ int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
>
>        /* Don't let a single process grow too big:
>           leave 3% of the size of this process for other processes */
> -       allowed -= current->mm->total_vm / 32;
> +       if (mm)
> +               allowed -= mm->total_vm / 32;
>
>        /*
>         * cast `allowed' as a signed long because vm_committed_space
> diff --git a/mm/shmem.c b/mm/shmem.c
> index d38d7e6..0ed0752 100644
> --- a/mm/shmem.c
> +++ b/mm/shmem.c
> @@ -161,8 +161,8 @@ static inline struct shmem_sb_info *SHMEM_SB(struct super_block *sb)
>  */
>  static inline int shmem_acct_size(unsigned long flags, loff_t size)
>  {
> -       return (flags & VM_ACCOUNT)?
> -               security_vm_enough_memory(VM_ACCT(size)): 0;
> +       return (flags & VM_ACCOUNT) ?
> +               security_vm_enough_memory_kern(VM_ACCT(size)) : 0;
>  }
>
>  static inline void shmem_unacct_size(unsigned long flags, loff_t size)
> @@ -179,8 +179,8 @@ static inline void shmem_unacct_size(unsigned long flags, loff_t size)
>  */
>  static inline int shmem_acct_block(unsigned long flags)
>  {
> -       return (flags & VM_ACCOUNT)?
> -               0: security_vm_enough_memory(VM_ACCT(PAGE_CACHE_SIZE));
> +       return (flags & VM_ACCOUNT) ?
> +               0 : security_vm_enough_memory_kern(VM_ACCT(PAGE_CACHE_SIZE));
>  }
>
>  static inline void shmem_unacct_blocks(unsigned long flags, long pages)
> diff --git a/security/security.c b/security/security.c
> index 255b085..c0acfa7 100644
> --- a/security/security.c
> +++ b/security/security.c
> @@ -198,14 +198,23 @@ int security_settime(struct timespec *ts, struct timezone *tz)
>
>  int security_vm_enough_memory(long pages)
>  {
> +       WARN_ON(current->mm == NULL);
>        return security_ops->vm_enough_memory(current->mm, pages);
>  }
>
>  int security_vm_enough_memory_mm(struct mm_struct *mm, long pages)
>  {
> +       WARN_ON(mm == NULL);
>        return security_ops->vm_enough_memory(mm, pages);
>  }
>
> +int security_vm_enough_memory_kern(long pages)
> +{
> +       /* If current->mm is a kernel thread then we will pass NULL,
> +          for this specific case that is fine */
> +       return security_ops->vm_enough_memory(current->mm, pages);
> +}
> +
>  int security_bprm_alloc(struct linux_binprm *bprm)
>  {
>        return security_ops->bprm_alloc_security(bprm);
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: TMPFS over NFSv4

[Greg KH]

On Tue, May 25, 2010 at 10:00:29AM +0100, Tharindu Rukshan Bamunuarachchi wrote:
> hope that the 2.6.27.48 or later will be shipped with SP1 :-)

What do you mean "SP1"?

confused,

greg k-h

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: TMPFS over NFSv4

[Greg KH]

On Mon, May 24, 2010 at 04:46:24PM -0700, Hugh Dickins wrote:
> Hi Greg,
> 
> On Mon, 24 May 2010, Alan Cox wrote:
> > On Mon, 24 May 2010 02:57:30 -0700
> > Hugh Dickins <hughd@google.com> wrote:
> > > On Mon, May 24, 2010 at 2:26 AM, Tharindu Rukshan Bamunuarachchi
> > > <btharindu@gmail.com> wrote:
> > > > thankx a lot Hugh ... I will try this out ... (bit harder patch
> > > > already patched SLES kernel :-p ) ....
> > > 
> > > If patch conflicts are a problem, you really only need to put in the
> > > two-liner patch to mm/mmap.c: Alan was seeking perfection in
> > > the rest of the patch, but you can get away without it.
> > > 
> > > >
> > > > BTW, what does Alan means by "strict overcommit" ?
> > > 
> > > Ah, that phrase, yes, it's a nonsense, but many of us do say it by mistake.
> > > Alan meant to say "strict no-overcommit".
> > 
> > No I always meant to say 'strict overcommit'. It avoids excess negatives
> > and "no noovercommit" discussions.
> > 
> > I guess 'strict overcommit control' would have been clearer 8)
> > 
> > Alan
> 
> I see we've just missed 2.6.27.47-rc1, but if there's to be an -rc2,
> please include Alan's 2.6.28 oops fix below: which Tharindu appears
> to be needing - just now discussed on linux-mm and linux-nfs.
> Failing that, please queue it up for 2.6.27.48.


There is now going to be a -rc2 due to other problems, so I'll go queue
this one up as well.

> Or if you'd prefer a smaller patch for -stable, then just the mm/mmap.c
> part of it should suffice: I think it's fair to say that the rest of the
> patch was more precautionary - as Alan describes, for catching other bugs,
> so good for an ongoing development tree, but not necessarily in -stable.
> (However, Alan may disagree - I've already misrepresented him once here!)

The original is best, it makes more sense.

thanks,

greg k-h

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>