[PATCH 1/1] mips: fault.c: Port OOM changes to do_page_fault

[PATCH 1/1] mips: fault.c: Port OOM changes to do_page_fault

[Kautuk Consul]

From: Kautuk Consul <consul.kautuk@gmail.com>

Commit d065bd810b6deb67d4897a14bfe21f8eb526ba99
(mm: retry page fault when blocking on disk transfer) and
commit 37b23e0525d393d48a7d59f870b3bc061a30ccdb
(x86,mm: make pagefault killable)

The above commits introduced changes into the x86 pagefault handler
for making the page fault handler retryable as well as killable.

These changes reduce the mmap_sem hold time, which is crucial
during OOM killer invocation.

Port these changes to MIPS.

Without these changes, my MIPS board encounters many hang and livelock
scenarios.
After applying this patch, OOM feature performance improves according to
my testing.

Signed-off-by: Mohd. Faris <mohdfarisq2010@gmail.com>
Signed-off-by: Kautuk Consul <consul.kautuk@gmail.com>
---
 arch/mips/mm/fault.c |   36 +++++++++++++++++++++++++++++-------
 1 files changed, 29 insertions(+), 7 deletions(-)

diff --git a/arch/mips/mm/fault.c b/arch/mips/mm/fault.c
index 937cf33..aae2cb3 100644
--- a/arch/mips/mm/fault.c
+++ b/arch/mips/mm/fault.c
@@ -42,6 +42,8 @@ asmlinkage void __kprobes do_page_fault(struct pt_regs *regs, unsigned long writ
 	const int field = sizeof(unsigned long) * 2;
 	siginfo_t info;
 	int fault;
+	unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE |
+						 (write ? FAULT_FLAG_WRITE : 0);
 
 #if 0
 	printk("Cpu%d[%s:%d:%0*lx:%ld:%0*lx]\n", raw_smp_processor_id(),
@@ -91,6 +93,7 @@ asmlinkage void __kprobes do_page_fault(struct pt_regs *regs, unsigned long writ
 	if (in_atomic() || !mm)
 		goto bad_area_nosemaphore;
 
+retry:
 	down_read(&mm->mmap_sem);
 	vma = find_vma(mm, address);
 	if (!vma)
@@ -144,7 +147,11 @@ good_area:
 	 * make sure we exit gracefully rather than endlessly redo
 	 * the fault.
 	 */
-	fault = handle_mm_fault(mm, vma, address, write ? FAULT_FLAG_WRITE : 0);
+	fault = handle_mm_fault(mm, vma, address, flags);
+
+	if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current))
+		return;
+
 	perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address);
 	if (unlikely(fault & VM_FAULT_ERROR)) {
 		if (fault & VM_FAULT_OOM)
@@ -153,12 +160,27 @@ good_area:
 			goto do_sigbus;
 		BUG();
 	}
-	if (fault & VM_FAULT_MAJOR) {
-		perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MAJ, 1, regs, address);
-		tsk->maj_flt++;
-	} else {
-		perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MIN, 1, regs, address);
-		tsk->min_flt++;
+	if (flags & FAULT_FLAG_ALLOW_RETRY) {
+		if (fault & VM_FAULT_MAJOR) {
+			perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MAJ, 1,
+						  regs, address);
+			tsk->maj_flt++;
+		} else {
+			perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MIN, 1,
+						  regs, address);
+			tsk->min_flt++;
+		}
+		if (fault & VM_FAULT_RETRY) {
+			flags &= ~FAULT_FLAG_ALLOW_RETRY;
+
+			/*
+			 * No need to up_read(&mm->mmap_sem) as we would
+			 * have already released it in __lock_page_or_retry
+			 * in mm/filemap.c.
+			 */
+
+			goto retry;
+		}
 	}
 
 	up_read(&mm->mmap_sem);
-- 
1.7.6

Re: [PATCH 1/1] mips: fault.c: Port OOM changes to do_page_fault

[Kautuk Consul]

[-- Attachment #1: Type: text/plain, Size: 755 bytes --]

>
> Without these changes, my MIPS board encounters many hang and livelock
> scenarios.
> After applying this patch, OOM feature performance improves according to
> my testing.
>

Just to clarify the hang scenario I am talking about, the test case I
used to reproduce
this problem is attached to this email.
Running this a few times hangs my console and then even Ctrl-C signals
do not get handled
and I also do not get back the command prompt.

After applying this patch things seem to improve, but there are still
some hangs which I
encounter which I am further trying to debug.

However, since the generic part of the kernel(mm/filemap.c) now
supports killable and
retryable page fault handling, I thought that this change would be
valid for MIPS too.

[-- Attachment #2: stress_32k.c --]
[-- Type: text/x-csrc, Size: 3897 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <pthread.h>
#include <unistd.h>

#define ALLOC_BYTE 512*1024
#define COUNT 50

void *alloc_function_one( void *ptr );
void *alloc_function_two( void *ptr );
void *alloc_function_three( void *ptr );
void *alloc_function_four( void *ptr );
void *alloc_function_five( void *ptr );
void *enable_function( void *ptr );


int main(int argc, char *argv[])
{
	pthread_t thread1, thread2, thread3, thread4, thread5;
	char *message1 = "Thread 1";
	char *message2 = "Thread 2";
	char *message3 = "Thread 3";
	char *message4 = "Thread 4";
	char *message5 = "Thread 5";
	int iret1 = -1;
	int iret2 = -1;
	int iret3 = -1;
	int iret4 = -1;
	int iret5 = -1;
	fork();
	iret1 = pthread_create( &thread1, NULL, alloc_function_one, (void*) message1);
	iret2 = pthread_create( &thread2, NULL, alloc_function_two, (void*) message2);
	iret2 = pthread_create( &thread3, NULL, alloc_function_three, (void*) message2);
	iret2 = pthread_create( &thread4, NULL, alloc_function_four, (void*) message2);
	iret2 = pthread_create( &thread5, NULL, alloc_function_five, (void*) message2);

	pthread_join( thread1, NULL);
	pthread_join( thread2, NULL);
	pthread_join( thread3, NULL);
	pthread_join( thread4, NULL);
	pthread_join( thread5, NULL);

	printf("Thread 1 returns: %d\n",iret1);
	printf("Thread 2 returns: %d\n",iret2);
	printf("Thread 3 returns: %d\n",iret3);
	printf("Thread 4 returns: %d\n",iret4);
	printf("Thread 5 returns: %d\n",iret5);
	exit(0);
}

void *alloc_function_two( void *ptr )
{
	char *message;
	message = (char *) ptr;
	void *myblock[COUNT];
	int i= 0,j=0;
	int freed=0;
	printf("message_alloc  \n");
	while(1)
	{
		memset(myblock,0,sizeof(myblock));
		printf("message_alloc %s \n",message);
		for(i=0;i< COUNT ;i++)
		{
			myblock[i] = (void *) malloc(ALLOC_BYTE);
			memset(myblock[i],1, ALLOC_BYTE);
		}
	}
}


void *alloc_function_one( void *ptr )
{
	char *message;
	message = (char *) ptr;
	void *myblock[COUNT];
	int i= 0,j=0;
	int freed=0;
	printf("message_alloc  \n");
	while(1)
	{
		memset(myblock,0,sizeof(myblock));
		printf("message_alloc %s \n",message);
		for(i=0;i< COUNT ;i++)
		{
			myblock[i] = (void *) malloc(ALLOC_BYTE);
			memset(myblock[i],1, ALLOC_BYTE);
		}
	}
}

void *alloc_function_three( void *ptr )
{
       char *message;
        message = (char *) ptr;
        void *myblock[COUNT];
        int i= 0,j=0;
        int freed=0;
        printf("message_alloc  \n");
        while(1)
        {
                memset(myblock,0,sizeof(myblock));
                printf("message_alloc %s \n",message);
                for(i=0;i< COUNT ;i++)
                {
                        myblock[i] = (void *) malloc(ALLOC_BYTE);
                        memset(myblock[i],1, ALLOC_BYTE);
                }
        }
}
void *alloc_function_four( void *ptr )
{
       char *message;
        message = (char *) ptr;
        void *myblock[COUNT];
        int i= 0,j=0;
        int freed=0;
        printf("message_alloc  \n");
        while(1)
        {
                memset(myblock,0,sizeof(myblock));
                printf("message_alloc %s \n",message);
                for(i=0;i< COUNT ;i++)
                {
                        myblock[i] = (void *) malloc(ALLOC_BYTE);
                        memset(myblock[i],1, ALLOC_BYTE);
                }
        }
}
void *alloc_function_five( void *ptr )
{
       char *message;
        message = (char *) ptr;
        void *myblock[COUNT];
        int i= 0,j=0;
        int freed=0;
        printf("message_alloc  \n");
        while(1)
        {
                memset(myblock,0,sizeof(myblock));
                printf("message_alloc %s \n",message);
                for(i=0;i< COUNT ;i++)
                {
                        myblock[i] = (void *) malloc(ALLOC_BYTE);
                        memset(myblock[i],1, ALLOC_BYTE);
                }
        }
}

Re: [PATCH 1/1] mips: fault.c: Port OOM changes to do_page_fault

[Kautuk Consul]

>
> However, since the generic part of the kernel(mm/filemap.c) now
> supports killable and
> retryable page fault handling, I thought that this change would be
> valid for MIPS too.


Did anyone get a chance to review this patch?

Re: [PATCH 1/1] mips: fault.c: Port OOM changes to do_page_fault

[Ralf Baechle]

On Fri, Dec 23, 2011 at 04:52:42PM +0530, Kautuk Consul wrote:

Applied.  Thanks Kautuk!

  Ralf