* Re: Cygwin can't handle huge packfiles?
From: Christopher Faylor @ 2006-04-06 0:34 UTC (permalink / raw)
To: git
In-Reply-To: <20060405232739.GA18121@nospam.com>
On Thu, Apr 06, 2006 at 01:27:39AM +0200, Rutger Nijlunsing wrote:
>On Wed, Apr 05, 2006 at 05:08:44PM -0400, Christopher Faylor wrote:
>> On Wed, Apr 05, 2006 at 04:14:20PM +0200, Johannes Schindelin wrote:
>> >> Inspired by a patch of Alex Riesen (thanks, Alex), I tried to use the
>> >> regular mmap for mapping pack files, only to discover that I compile
>> >> without defining "NO_MMAP", so I've been using the stock mmap all
>> >> along. So now I'm thinking that the cygwin mmap also does a
>> >> malloc-and-read, just like git does with NO_MMAP. So I'll continue to
>> >> investigate in that direction.
>> >
>> >I think cygwin's mmap() is based on the Win32 API equivalent, which could
>> >mean that it *is* memory mapped, but in a special area (which is smaller
>> >than 1.5 gigabyte). In this case, it would make sense to limit the pack
>> >size, thereby having several packs, and mmap() them as they are needed.
>>
>> Yes, cygwin's mmap uses CreateFileMapping and MapViewOfFile. IIRC,
>> Windows might have a 2G limitation lurking under the hood somewhere but
>> I think that might be tweakable with some registry setting.
>
>Windows places its DLLs criss-cross through the memory space because
>every DLL on the system has its own preferred place to be loaded (the
>base address). This severely limits the amount of largest contiguous
>memory block available, which is needed for one mmap() I think.
>
>Several solutions exist:
> - enlarge the address space with the /3GB boot flag in boot.ini
Thanks. The 3GB boot flag is what I was trying to remember.
> - rebase all DLLs with REBASE.EXE (part of platform sdk) .
> Just make them the same and fix them to a low address.
> Problem is rebasing system dlls since those are locked by the system.
Cygwin has its own version of rebase and a method for rebasing all of the
dlls in the distribution. Using that may help squeeze out a little bit
of memory.
> - at start of program before other DLLs are loaded,
> reserve an as large part of the memory as possible with
> VirtualAlloc()
Cygwin actually uses this trick to try to push DLLs into their right
locations after a fork. It sort of works but sometimes, in a child
proccess, Windows puts "stuff" in locations previously occupied by a
DLL. I could swear that it does that just to be annoying...
There is a chicken/egg problem here in that Cygwin uses Doug Lea's malloc
and that version of malloc will use mmap when sbrk() fails -- as it is
apt to do when allocating gigabytes of memory. So, using malloc is
not a way to avoid mmap.
cgf
^ permalink raw reply
* Re: Q on audit, audit-syscall
From: Steve Grubb @ 2006-04-06 0:34 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-audit
In-Reply-To: <20060405153716.6f71939f.akpm@osdl.org>
On Wednesday 05 April 2006 18:37, Andrew Morton wrote:
> Herbert Rosmanith <kernel@wildsau.enemy.org> wrote:
> > (2) in linux/Documentation/devices.txt I've found an "audit device":
> >
> > 103 block Audit device
> > 0 = /dev/audit Audit device
>
> hm. I don't think that's true, is it? If not, can we clean
> it up please?
Hi Andrew,
This looks like some Suse documentation. We've never touched the devices.txt
file since...well...we don't use any devices. But the documentation may be
valid since that may be what they use. You'd probably need to ask someone
that maintains Laus if the documentation is for their audit system and
accurate.
If the devices file is to document only the accepted - in the 2.6 kernel.org
kernel, then we should delete that reference. If its to document what has
been approved, even if its proprietary, then it should be clarified that its
Laus and not the native 2.6 kernel audit system.
Hope this helps...
-Steve
^ permalink raw reply
* Re: udev 085 warnings
From: Michael Buesch @ 2006-04-06 0:30 UTC (permalink / raw)
To: linux-hotplug
In-Reply-To: <Pine.LNX.4.61.0604051110170.11179@yvahk01.tjqt.qr>
[-- Attachment #1: Type: text/plain, Size: 1237 bytes --]
On Wednesday 05 April 2006 21:03, Kay Sievers wrote:
> On Wed, Apr 05, 2006 at 11:10:40AM +0200, Jan Engelhardt wrote:
> > I am currently compiling udev-085-14.src.rpm from the opensuse
> > OSS-factory tree on a 64-bit arch and get a lot of these warnings:
> >
> > fat.c:325: warning: cast increases required alignment of target type
> > 325:
> > next = le32_to_cpu(*((uint32_t *) buf) & 0x0fffffff);
Hm, I don't know what this code is about, but is it actually correct?
I mean, it masks the value of buf, which is appearently a little-endian
32bit value with 0x0fffffff (which is CPU order). Is that intentional?
Or should it be something like that instead:
next = le32_to_cpu(*((uint32_t *) buf)) & 0x0fffffff;
> > Will there be a proper fix or is this just another of these nice gcc
> > warnings that are superfluous?
>
> GCC is correct warning about that, cause the Makefile asked for that
> kind of warning. The buffer access is always aligned but the compiler
> can't know that. To get rid of it, you would need to memcpy() the integer
> in a variable or access it byte by byte and shift it to an integer
> value. Removing -Wcast-align should be the easiest. :)
--
Greetings Michael.
[-- Attachment #2: Type: application/pgp-signature, Size: 191 bytes --]
^ permalink raw reply
* Re: Error getting VM-informations with Python on Xen 3.0.2
From: Ewan Mellor @ 2006-04-06 0:27 UTC (permalink / raw)
To: Thorolf Godawa; +Cc: xen-devel
In-Reply-To: <4434197A.8000308@godawa.de>
On Wed, Apr 05, 2006 at 09:24:42PM +0200, Thorolf Godawa wrote:
> Hi all,
>
> I wrote a small python-script to monitor the status of the started VMs
> based on the xenmon.py and the following code I got from Anthony Liguori:
>
> [Snip]
>
> It breaks at line "doms = server.xend_list_domains()" with the following
> error:
>
> ----------------------------------------
> Traceback (most recent call last):
> File "xentest1.py", line 22, in ?
> doms = server.xend_list_domains()
> File "/usr/lib/python2.4/xmlrpclib.py", line 1096, in __call__
> return self.__send(self.__name, args)
> File "/usr/lib/python2.4/xmlrpclib.py", line 1383, in __request
> verbose=self.__verbose
> File "/usr/lib/python2.4/site-packages/xen/util/xmlrpclib2.py", line
> 46, in request
> return Transport.request(self, host, '/RPC2', request_body, verbose)
> File "/usr/lib/python2.4/xmlrpclib.py", line 1147, in request
> return self._parse_response(h.getfile(), sock)
> File "/usr/lib/python2.4/xmlrpclib.py", line 1286, in _parse_response
> return u.close()
> File "/usr/lib/python2.4/xmlrpclib.py", line 744, in close
> raise Fault(**self._stack[0])
> xmlrpclib.Fault: <Fault 1: 'Traceback (most recent call last):\n File
> "/usr/lib/python2.4/site-packages/xen/util/xmlrpclib2.py", line 74, in
> _marshaled_dispatch\n response = self._dispatch(method, params)\n
> File "/usr/lib/python2.4/SimpleXMLRPCServer.py", line 408, in
> _dispatch\n raise Exception(\'method "%s" is not supported\' %
> method)\nException: method "xend_list_domains" is not supported\n'>
> ----------------------------------------
>
> If I see right it now uses "/usr/lib/python2.4/SimpleXMLRPCServer.py"
> (what does not exist in Xen 3.0_8659-2) and breaks because the method
> "xend_list_domains" is not supported anymore I use to get the
> domain-informations with the script.
>
> Is there a better way to get the VM-status like name, memory, state,
> CPU-usage etc. via Python than using the mechanism from xenmon.py and
> the above one?
Try server.xend.domains(True). This changed with the new XML-RPC interface,
courtesy of the aforementioned hacker.
Ewan.
^ permalink raw reply
* [HVM]{PATCH] Fix localtime handling for fully virtualized guests
From: Bruce Rogers @ 2006-04-06 0:26 UTC (permalink / raw)
To: xen-devel
[-- Attachment #1: Type: text/plain, Size: 143 bytes --]
Default value was wrong, causing the fully virtualized guests to always get localtime.
Signed-off-by: Bruce Rogers <brogers@novell.com
[-- Attachment #2: hvm-localtime-fix.patch --]
[-- Type: text/plain, Size: 337 bytes --]
diff -r e1152d55ea31 tools/ioemu/vl.c
--- a/tools/ioemu/vl.c Wed Apr 5 16:41:51 2006
+++ b/tools/ioemu/vl.c Wed Apr 5 18:20:28 2006
@@ -138,7 +138,7 @@
int gus_enabled = 1;
int pci_enabled = 1;
int prep_enabled = 0;
-int rtc_utc = 0;
+int rtc_utc = 1;
int cirrus_vga_enabled = 1;
int vga_accelerate = 1;
int graphic_width = 800;
[-- Attachment #3: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply
* Re: readers-writers mutex
From: Jeff V. Merkey @ 2006-04-06 1:15 UTC (permalink / raw)
To: Joshua Hudson; +Cc: linux-kernel
In-Reply-To: <bda6d13a0604051521o229de77dvb38992d6427a450c@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2468 bytes --]
Joshua Hudson wrote:
>Since we are moving from semaphores to mutex, there should be a
>mutex_rw. I had a try
>at creating one (might as well convert from sem_rw). I'll bet somebody
>here can do a
>lot better, but this will do if need be.
>
>--- linux-2.6.16.1-stock/include/linux/mutex_rw.h 1969-12-31
>16:00:00.000000000 -0800
>+++ linux-2.6.16.1-nvl/include/linux/mutex_rw.h 2006-04-04
>18:11:56.000000000 -0700
>@@ -0,0 +1,60 @@
>+/* Linux RW mutex
>+ * This file: GNU GPL v2 or later, Joshua Hudson <joshudson@gmail.com>
>+ *
>+ * Somebody else can make this fast. I just made this work.
>+ *
>+ * DANGER! Change of this file will break module binaries if any
>+ * rw mutex is shared between main kernel and modules or between
>+ * modules with a different version.
>+ */
>+
>+#ifndef __KERNEL_MUTEX_RW
>+#define __KERNEL_MUTEX_RW
>+#ifdef __KERNEL__
>+
>+#include <linux/mutex.h>
>+
>+struct rw_mutex {
>+ struct mutex r_mutex;
>+ struct mutex w_mutex;
>+ unsigned n_readers;
>+};
>+
>+static inline void rw_mutex_init(struct rw_mutex *rw)
>+{
>+ mutex_init(&rw->r_mutex);
>+ mutex_init(&rw->w_mutex);
>+ rw->n_readers = 0;
>+}
>+
>+static inline void rw_mutex_destroy(struct rw_mutex *rw)
>+{
>+ mutex_destroy(&rw->r_mutex);
>+ mutex_destroy(&rw->w_mutex);
>+}
>+
>+static inline void mutex_lock_w(struct rw_mutex *rw)
>+{
>+ mutex_lock(&rw->w_mutex);
>+}
>+
>+static inline void mutex_unlock_w(struct rw_mutex *rw)
>+{
>+ mutex_unlock(&rw->w_mutex);
>+}
>+
>+static inline void mutex_lock_r(struct rw_mutex *rw)
>+{
>+ mutex_lock(&rw->r_mutex);
>+ if (++rw->n_readers == 1)
>+ mutex_lock(&rw->w_mutex);
>+ mutex_unlock(&rw->r_mutex);
>+}
>+
>+static inline void mutex_unlock_r(struct rw_mutex *rw)
>+{
>+ mutex_lock(&rw->r_mutex);
>+ if (--rw->n_readers == 0)
>+ mutex_unlock(&rw->w_mutex);
>+ mutex_unlock(&rw->r_mutex);
>+}
>+
>+#endif
>+#endif
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at http://www.tux.org/lkml/
>
>
>
Race conditions here.
Try this code. These have been tested and I have been using the code for
couple of years in another OS (which is just about done now and is in
alpha and beta and little more SMP friendly).
Jeff
[-- Attachment #2: rwlock.c --]
[-- Type: text/x-csrc, Size: 22909 bytes --]
/***************************************************************************
*
* Copyright (c) 2005, 2006 Cherokee Nation. All Rights Reserved.
*
* This Program is released under GPLv3
*
* AUTHOR : Jeff V. Merkey
* FILE : RWLOCK.C
* DESCRIP : Multi-Processing Reader/Writer Locks Gadugi v7
* DATE : October 23, 2004
*
***************************************************************************/
#include "stdarg.h"
#include "stdio.h"
#include "stdlib.h"
#include "ctype.h"
#include "string.h"
#include "kernel.h"
#include "keyboard.h"
#include "screen.h"
#include "types.h"
#include "emit.h"
#include "dos.h"
#include "tss.h"
#include "os.h"
#include "mps.h"
#include "hal.h"
#include "timer.h"
#include "peexe.h"
#include "malloc.h"
#include "free.h"
#include "event.h"
#define WRITER_OWNED -10
#define READER_OWNED -11
#define UNOWNED -12
rwlock_t *rwlock_init(rwlock_t *rwlock);
rwlock_t *rwlock_alloc(void);
LONG rwlock_free(rwlock_t *rwlock);
LONG rwlock_read_lock(rwlock_t *rwlock);
LONG rwlock_write_lock(rwlock_t *rwlock);
LONG rwlock_read_try_lock(rwlock_t *rwlock);
LONG rwlock_write_try_lock(rwlock_t *rwlock);
LONG rwlock_unlock(rwlock_t *rwlock);
LONG rwlock_reader_to_writer(rwlock_t *rwlock);
LONG rwlock_reader_to_writer_unordered(rwlock_t *rwlock);
LONG rwlock_writer_to_reader(rwlock_t *rwlock);
LONG rwlock_release(rwlock_t *rwlock);
PROCESS *get_rwlock_process(rwlock_t *rwlock);
void put_rwlock_process(rwlock_t *rwlock, PROCESS *p);
rwlock_t *rwlock_init(rwlock_t *rwlock)
{
if (rwlock && !rwlock->signature)
{
SetData((LONG *) rwlock, 0, sizeof(rwlock_t));
rwlock->signature = LRWLOCK_SIGNATURE;
rwlock->flags = UNOWNED;
return (rwlock_t *) rwlock;
}
return 0;
}
rwlock_t *rwlock_alloc(void)
{
register rwlock_t *rwlock;
rwlock = kmalloc(sizeof(rwlock_t));
if (rwlock)
{
SetData((LONG *) rwlock, 0, sizeof(rwlock_t));
rwlock->signature = RWLOCK_SIGNATURE;
rwlock->flags = UNOWNED;
return (rwlock_t *) rwlock;
}
return 0;
}
LONG rwlock_free(rwlock_t *rwlock)
{
if (rwlock->signature == RWLOCK_SIGNATURE ||
rwlock->signature == LRWLOCK_SIGNATURE)
{
if (!rwlock_release(rwlock))
{
if (rwlock->signature == RWLOCK_SIGNATURE)
kfree(rwlock);
else
rwlock->signature = 0;
return 0;
}
return -2;
}
return -1;
}
#if RWLOCK_TRACE
BYTE *get_RW_TYPE(LONG p)
{
switch (p)
{
case RREAD_LOCK:
return "READ_LOCK";
case RREAD_UNLOCK:
return "READ_UNLOCK";
case WWRITE_LOCK:
return "WRITE_LOCK";
case WWRITE_UNLOCK:
return "WRITE_UNLOCK";
default:
return "????";
}
}
BYTE *get_RW_OP(LONG p)
{
switch (p)
{
case READER_LOCK:
return "READER_LOCK ";
case READER_UNLOCK:
return "READER_UNLOCK";
case READER_SLEEP:
return "READER_SLEEP ";
case READER_WAKE:
return "READER_WAKE ";
case WRITER_LOCK:
return "WRITER_LOCK ";
case WRITER_UNLOCK:
return "WRITER_UNLOCK";
case WRITER_SLEEP:
return "WRITER_SLEEP ";
case WRITER_WAKE:
return "WRITER_WAKE ";
default:
return "???? ";
}
}
void RW_DISPLAY(SCREEN *screen, rwlock_t *rwlock)
{
register LONG i;
SetPauseMode(screen, screen->nLines - 5);
printfScreen(screen, "RWIndex %d\n");
for (i=0; i < 256; i++)
{
printfScreen(screen, "process-%08X time-%08X target-%08X s-%s t-%s\n",
rwlock->RWProcess[i],
rwlock->RWTime[i],
rwlock->RWTarget[i],
get_RW_OP(rwlock->RWOp[i]),
get_RW_TYPE(rwlock->RWType[i]));
}
ClearPauseMode(screen);
return;
}
void RW_TRACE(rwlock_t *rwlock, LONG event, LONG target, LONG type)
{
LONG base, counter;
GetSystemDTSC(&base, &counter);
rwlock->RWProcess[rwlock->RWIndex & 0xFF] = (LONG) get_running_process();
rwlock->RWOp[rwlock->RWIndex & 0xFF] = event;
rwlock->RWTarget[rwlock->RWIndex & 0xFF] = target;
rwlock->RWType[rwlock->RWIndex & 0xFF] = type;
rwlock->RWTime[rwlock->RWIndex & 0xFF] = (LONG) base;
rwlock->RWIndex++;
}
#endif
LONG rwlock_read_lock(rwlock_t *rwlock)
{
register PROCESS *p = get_running_process();
register LONG flags;
if (rwlock->signature == RWLOCK_SIGNATURE ||
rwlock->signature == LRWLOCK_SIGNATURE)
{
flags = get_flags();
spin_lock(&rwlock->mutex);
p->syncFlag = 0;
switch (rwlock->flags)
{
case WRITER_OWNED:
#if RWLOCK_TRACE
RW_TRACE(rwlock, READER_SLEEP, (LONG) p, RREAD_LOCK);
#endif
spin_lock(&p->threadMutex); // lock current thread
put_rwlock_process(rwlock, p);
p->stackPointer = 0;
p->threadState = PS_SLEEP;
p->syncObject = rwlock;
p->syncState = PROCESS_READER_BLOCKED_SYNC;
spin_unlock(&rwlock->mutex);
thread_switch(); // context switch will unlock thread
cli();
set_flags(flags);
return p->syncFlag;
case READER_OWNED:
if (rwlock->writers)
{
#if RWLOCK_TRACE
RW_TRACE(rwlock, READER_SLEEP, (LONG) p, RREAD_LOCK);
#endif
spin_lock(&p->threadMutex); // lock current thread
put_rwlock_process(rwlock, p);
p->stackPointer = 0;
p->threadState = PS_SLEEP;
p->syncObject = rwlock;
p->syncState = PROCESS_READER_BLOCKED_SYNC;
spin_unlock(&rwlock->mutex);
thread_switch(); // context switch will unlock thread
cli();
set_flags(flags);
return p->syncFlag;
}
#if RWLOCK_TRACE
RW_TRACE(rwlock, READER_LOCK, 0, RREAD_LOCK);
#endif
rwlock->flags = (LONG) READER_OWNED;
rwlock->readers++; // set reader count
spin_unlock(&rwlock->mutex);
set_flags(flags);
return p->syncFlag;
case UNOWNED:
#if RWLOCK_TRACE
RW_TRACE(rwlock, READER_LOCK, 0, RREAD_LOCK);
#endif
rwlock->flags = (LONG) READER_OWNED;
rwlock->readers++; // set reader count
spin_unlock(&rwlock->mutex);
set_flags(flags);
return p->syncFlag;
default:
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
}
}
return -1;
}
LONG rwlock_write_lock(rwlock_t *rwlock)
{
register PROCESS *p = get_running_process();
register LONG flags;
if (rwlock->signature == RWLOCK_SIGNATURE ||
rwlock->signature == LRWLOCK_SIGNATURE)
{
flags = get_flags();
spin_lock(&rwlock->mutex);
p->syncFlag = 0;
switch (rwlock->flags)
{
case READER_OWNED:
case WRITER_OWNED:
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_SLEEP, (LONG) p, WWRITE_LOCK);
#endif
spin_lock(&p->threadMutex); // lock current thread
put_rwlock_process(rwlock, p);
rwlock->writers++;
p->stackPointer = 0;
p->threadState = PS_SLEEP;
p->syncObject = rwlock;
p->syncState = PROCESS_WRITER_BLOCKED_SYNC;
spin_unlock(&rwlock->mutex);
thread_switch(); // context switch will unlock thread
cli();
set_flags(flags);
return p->syncFlag;
case UNOWNED:
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_LOCK, 0, WWRITE_LOCK);
#endif
rwlock->flags = (LONG) WRITER_OWNED;
rwlock->owner = (LONG) p;
spin_unlock(&rwlock->mutex);
set_flags(flags);
return p->syncFlag;
default:
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
}
}
return -1;
}
LONG rwlock_read_try_lock(rwlock_t *rwlock)
{
register PROCESS *p = get_running_process();
register LONG flags;
if (rwlock->signature == RWLOCK_SIGNATURE ||
rwlock->signature == LRWLOCK_SIGNATURE)
{
flags = get_flags();
spin_lock(&rwlock->mutex);
p->syncFlag = 0;
switch (rwlock->flags)
{
case WRITER_OWNED:
spin_unlock(&rwlock->mutex);
set_flags(flags);
return 1;
case READER_OWNED:
if (rwlock->writers)
{
spin_unlock(&rwlock->mutex);
set_flags(flags);
return 1;
}
#if RWLOCK_TRACE
RW_TRACE(rwlock, READER_LOCK, 0, RREAD_TRY_LOCK);
#endif
rwlock->flags = (LONG) READER_OWNED;
rwlock->readers++; // set reader count
spin_unlock(&rwlock->mutex);
set_flags(flags);
return p->syncFlag;
case UNOWNED:
#if RWLOCK_TRACE
RW_TRACE(rwlock, READER_LOCK, 0, RREAD_TRY_LOCK);
#endif
rwlock->flags = (LONG) READER_OWNED;
rwlock->readers++; // set reader count
spin_unlock(&rwlock->mutex);
set_flags(flags);
return p->syncFlag;
default:
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
}
}
return -1;
}
LONG rwlock_write_try_lock(rwlock_t *rwlock)
{
register PROCESS *p = get_running_process();
register LONG flags;
if (rwlock->signature == RWLOCK_SIGNATURE ||
rwlock->signature == LRWLOCK_SIGNATURE)
{
flags = get_flags();
spin_lock(&rwlock->mutex);
p->syncFlag = 0;
switch (rwlock->flags)
{
case READER_OWNED:
case WRITER_OWNED:
spin_unlock(&rwlock->mutex);
set_flags(flags);
return 1;
case UNOWNED:
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_LOCK, 0, WWRITE_TRY_LOCK);
#endif
rwlock->flags = (LONG) WRITER_OWNED;
rwlock->owner = (LONG) p;
spin_unlock(&rwlock->mutex);
set_flags(flags);
return p->syncFlag;
default:
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
}
}
return -1;
}
LONG rwlock_read_unlock(rwlock_t *rwlock)
{
register PROCESS *p;
register LONG flags;
if (rwlock->signature == RWLOCK_SIGNATURE ||
rwlock->signature == LRWLOCK_SIGNATURE)
{
flags = get_flags();
spin_lock(&rwlock->mutex);
switch (rwlock->flags)
{
case WRITER_OWNED:
panic("rwlock_read_unlock called with writer owning rwlock");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
case READER_OWNED:
if (rwlock->readers)
rwlock->readers--;
#if RWLOCK_TRACE
RW_TRACE(rwlock, READER_UNLOCK, 0, RREAD_UNLOCK);
#endif
if (!rwlock->readers)
{
rwlock->flags = UNOWNED;
if (rwlock->head)
{
p = rwlock->head;
if (p->syncState != PROCESS_WRITER_BLOCKED_SYNC)
{
panic("rwlock_read_unlock called with sleeping READER");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
}
p = get_rwlock_process(rwlock);
rwlock->writers--;
spin_lock(&p->threadMutex); // lock thread
if (p->threadState == PS_SLEEP)
{
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_WAKE, (LONG) p, RREAD_UNLOCK);
#endif
p->threadState = PS_ACTIVE;
p->syncObject = 0;
p->syncState = 0;
p->syncFlag = 0;
(!p->processorBinding)
? put_dispatch(p)
: put_bind_pset(p, (PROCESSOR_SET *)p->processorBinding);
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_LOCK, 0, RREAD_UNLOCK);
#endif
rwlock->flags = WRITER_OWNED;
rwlock->owner = (LONG) p;
}
spin_unlock(&p->threadMutex); // unlock thread
}
}
spin_unlock(&rwlock->mutex);
set_flags(flags);
return 0;
case UNOWNED:
default:
panic("rwlock_read_unlock called in inconsistent state");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
}
}
return -1;
}
LONG rwlock_write_unlock(rwlock_t *rwlock)
{
register PROCESS *p, *curr;
register LONG flags;
if (rwlock->signature == RWLOCK_SIGNATURE ||
rwlock->signature == LRWLOCK_SIGNATURE)
{
flags = get_flags();
spin_lock(&rwlock->mutex);
curr = get_running_process();
if (rwlock->owner != (LONG) curr)
{
panic("rwlock_write_unlock called by process other than owner");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
}
switch (rwlock->flags)
{
case WRITER_OWNED:
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_UNLOCK, 0, WWRITE_UNLOCK);
#endif
rwlock->owner = 0;
rwlock->flags = UNOWNED;
if (rwlock->head)
{
p = rwlock->head;
if (p->syncState == PROCESS_WRITER_BLOCKED_SYNC)
{
p = get_rwlock_process(rwlock);
rwlock->writers--;
spin_lock(&p->threadMutex); // lock thread
if (p->threadState == PS_SLEEP)
{
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_WAKE, (LONG) p, WWRITE_UNLOCK);
#endif
p->threadState = PS_ACTIVE;
p->syncObject = 0;
p->syncState = 0;
p->syncFlag = 0;
(!p->processorBinding)
? put_dispatch(p)
: put_bind_pset(p, (PROCESSOR_SET *)p->processorBinding);
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_LOCK, 0, WWRITE_UNLOCK);
#endif
rwlock->flags = WRITER_OWNED;
rwlock->owner = (LONG) p;
}
spin_unlock(&p->threadMutex); // unlock thread
spin_unlock(&rwlock->mutex);
set_flags(flags);
return 0;
}
while (rwlock->head)
{
p = rwlock->head;
if (p->syncState == PROCESS_WRITER_BLOCKED_SYNC)
break;
p = get_rwlock_process(rwlock);
spin_lock(&p->threadMutex); // lock thread
if (p->threadState == PS_SLEEP)
{
#if RWLOCK_TRACE
RW_TRACE(rwlock, READER_WAKE, (LONG) p, WWRITE_UNLOCK);
#endif
p->threadState = PS_ACTIVE;
p->syncObject = 0;
p->syncState = 0;
p->syncFlag = 0;
(!p->processorBinding)
? put_dispatch(p)
: put_bind_pset(p, (PROCESSOR_SET *)p->processorBinding);
#if RWLOCK_TRACE
RW_TRACE(rwlock, READER_LOCK, 0, WWRITE_UNLOCK);
#endif
rwlock->flags = READER_OWNED;
rwlock->readers++;
rwlock->owner = 0; }
}
spin_unlock(&p->threadMutex); // unlock thread
}
spin_unlock(&rwlock->mutex);
set_flags(flags);
return 0;
case READER_OWNED:
panic("rwlock_write_unlock called with reader owning rwlock");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
case UNOWNED:
default:
panic("rwlock_write_unlock called in inconsistent state");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
}
}
return -1;
}
LONG rwlock_reader_to_writer(rwlock_t *rwlock)
{
register PROCESS *p;
register LONG flags;
if (rwlock->signature == RWLOCK_SIGNATURE ||
rwlock->signature == LRWLOCK_SIGNATURE)
{
flags = get_flags();
spin_lock(&rwlock->mutex);
switch (rwlock->flags)
{
case WRITER_OWNED:
panic("rwlock_reader_to_writer called with writer owning rwlock");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
case READER_OWNED:
if (rwlock->readers)
rwlock->readers--;
#if RWLOCK_TRACE
RW_TRACE(rwlock, READER_UNLOCK, 0, RREAD_UNLOCK);
#endif
if (!rwlock->readers)
{
rwlock->flags = UNOWNED;
if (rwlock->head)
{
p = rwlock->head;
if (p->syncState != PROCESS_WRITER_BLOCKED_SYNC)
{
panic("rwlock_read_unlock called with sleeping READER");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
}
p = get_rwlock_process(rwlock);
rwlock->writers--;
spin_lock(&p->threadMutex); // lock thread
if (p->threadState == PS_SLEEP)
{
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_WAKE, (LONG) p, RREAD_UNLOCK);
#endif
p->threadState = PS_ACTIVE;
p->syncObject = 0;
p->syncState = 0;
p->syncFlag = 0;
(!p->processorBinding)
? put_dispatch(p)
: put_bind_pset(p, (PROCESSOR_SET *)p->processorBinding);
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_LOCK, 0, RREAD_UNLOCK);
#endif
rwlock->flags = WRITER_OWNED;
rwlock->owner = (LONG) p;
}
spin_unlock(&p->threadMutex); // unlock thread
}
}
spin_unlock(&rwlock->mutex);
set_flags(flags);
return 0;
case UNOWNED:
default:
panic("rwlock_reader_to_writer called in inconsistent state");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
}
}
return -1;
}
LONG rwlock_reader_to_writer_unordered(rwlock_t *rwlock)
{
register PROCESS *p;
register LONG flags;
if (rwlock->signature == RWLOCK_SIGNATURE ||
rwlock->signature == LRWLOCK_SIGNATURE)
{
flags = get_flags();
spin_lock(&rwlock->mutex);
switch (rwlock->flags)
{
case WRITER_OWNED:
panic("rwlock_reader_to_writer_unordered called with writer owning rwlock");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
case READER_OWNED:
if (rwlock->readers)
rwlock->readers--;
#if RWLOCK_TRACE
RW_TRACE(rwlock, READER_UNLOCK, 0, RREAD_UNLOCK);
#endif
if (!rwlock->readers)
{
rwlock->flags = UNOWNED;
if (rwlock->head)
{
p = rwlock->head;
if (p->syncState != PROCESS_WRITER_BLOCKED_SYNC)
{
panic("rwlock_read_unlock called with sleeping READER");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
}
p = get_rwlock_process(rwlock);
rwlock->writers--;
spin_lock(&p->threadMutex); // lock thread
if (p->threadState == PS_SLEEP)
{
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_WAKE, (LONG) p, RREAD_UNLOCK);
#endif
p->threadState = PS_ACTIVE;
p->syncObject = 0;
p->syncState = 0;
p->syncFlag = 0;
(!p->processorBinding)
? put_dispatch(p)
: put_bind_pset(p, (PROCESSOR_SET *)p->processorBinding);
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_LOCK, 0, RREAD_UNLOCK);
#endif
rwlock->flags = WRITER_OWNED;
rwlock->owner = (LONG) p;
}
spin_unlock(&p->threadMutex); // unlock thread
}
}
spin_unlock(&rwlock->mutex);
set_flags(flags);
return 0;
case UNOWNED:
default:
panic("rwlock_reader_to_writer_unordered called in inconsistent state");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
}
}
return -1;
}
LONG rwlock_writer_to_reader(rwlock_t *rwlock)
{
register PROCESS *p, *curr;
register LONG flags;
if (rwlock->signature == RWLOCK_SIGNATURE ||
rwlock->signature == LRWLOCK_SIGNATURE)
{
flags = get_flags();
spin_lock(&rwlock->mutex);
curr = get_running_process();
if (rwlock->owner != (LONG) curr)
{
panic("rwlock_writer_to_reader called by process other than owner");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
}
switch (rwlock->flags)
{
case WRITER_OWNED:
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_UNLOCK, 0, WWRITE_UNLOCK);
#endif
rwlock->owner = 0;
rwlock->flags = UNOWNED;
if (rwlock->head)
{
p = rwlock->head;
if (p->syncState == PROCESS_WRITER_BLOCKED_SYNC)
{
p = get_rwlock_process(rwlock);
rwlock->writers--;
spin_lock(&p->threadMutex); // lock thread
if (p->threadState == PS_SLEEP)
{
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_WAKE, (LONG) p, WWRITE_UNLOCK);
#endif
p->threadState = PS_ACTIVE;
p->syncObject = 0;
p->syncState = 0;
p->syncFlag = 0;
(!p->processorBinding)
? put_dispatch(p)
: put_bind_pset(p, (PROCESSOR_SET *)p->processorBinding);
#if RWLOCK_TRACE
RW_TRACE(rwlock, WRITER_LOCK, 0, WWRITE_UNLOCK);
#endif
rwlock->flags = WRITER_OWNED;
rwlock->owner = (LONG) p;
}
spin_unlock(&p->threadMutex); // unlock thread
spin_unlock(&rwlock->mutex);
set_flags(flags);
return 0;
}
while (rwlock->head)
{
p = rwlock->head;
if (p->syncState == PROCESS_WRITER_BLOCKED_SYNC)
break;
p = get_rwlock_process(rwlock);
spin_lock(&p->threadMutex); // lock thread
if (p->threadState == PS_SLEEP)
{
#if RWLOCK_TRACE
RW_TRACE(rwlock, READER_WAKE, (LONG) p, WWRITE_UNLOCK);
#endif
p->threadState = PS_ACTIVE;
p->syncObject = 0;
p->syncState = 0;
p->syncFlag = 0;
(!p->processorBinding)
? put_dispatch(p)
: put_bind_pset(p, (PROCESSOR_SET *)p->processorBinding);
#if RWLOCK_TRACE
RW_TRACE(rwlock, READER_LOCK, 0, WWRITE_UNLOCK);
#endif
rwlock->flags = READER_OWNED;
rwlock->readers++;
rwlock->owner = 0; }
}
spin_unlock(&p->threadMutex); // unlock thread
}
spin_unlock(&rwlock->mutex);
set_flags(flags);
return 0;
case READER_OWNED:
panic("rwlock_writer_to_reader called with reader owning rwlock");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
case UNOWNED:
default:
panic("rwlock_writer_to_reader called in inconsistent state");
spin_unlock(&rwlock->mutex);
set_flags(flags);
return -1;
}
}
return -1;
}
PROCESS *get_rwlock_process(rwlock_t *rwlock)
{
register PROCESS *p, *list;
if (rwlock->head)
{
p = rwlock->head;
list = (PROCESS *) rwlock->head = (void *) p->syncNext;
if (list)
list->syncPrior = NULL;
else
rwlock->tail = NULL;
return p;
}
return 0;
}
void put_rwlock_process(rwlock_t *rwlock, PROCESS *p)
{
register PROCESS *search, *list;
search = rwlock->head;
while (search)
{
if (search == p)
return;
search = search->syncNext;
}
if (!rwlock->head)
{
rwlock->head = rwlock->tail = p;
p->syncNext = p->syncPrior = 0;
}
else
{
list = (PROCESS *) rwlock->tail;
list->syncNext = p;
p->syncNext = 0;
p->syncPrior = rwlock->tail;
rwlock->tail = p;
}
return;
}
LONG rwlock_release(rwlock_t *rwlock)
{
register PROCESS *p, *list;
register LONG flags;
if (rwlock->signature == RWLOCK_SIGNATURE ||
rwlock->signature == LRWLOCK_SIGNATURE)
{
flags = get_flags();
spin_lock(&rwlock->mutex);
p = rwlock->head;
while (p)
{
list = p->syncNext;
spin_lock(&p->threadMutex); // lock thread
if (p->threadState == PS_SLEEP)
{
p->threadState = PS_ACTIVE;
p->syncObject = 0;
p->syncState = 0;
p->syncFlag = -2;
if (rwlock->waiters)
rwlock->waiters--;
(!p->processorBinding)
? put_dispatch(p)
: put_bind_pset(p, (PROCESSOR_SET *)p->processorBinding);
}
spin_unlock(&p->threadMutex); // unlock thread
p = list;
}
rwlock->owner = 0;
spin_unlock(&rwlock->mutex);
set_flags(flags);
return 0;
}
return -1;
}
^ permalink raw reply
* Re: [patch 2/3] mm: speculative get_page
From: Nick Piggin @ 2006-04-05 0:27 UTC (permalink / raw)
To: Christoph Lameter
Cc: Nick Piggin, Andrew Morton, Linux Kernel, Linux Memory Management
In-Reply-To: <Pine.LNX.4.64.0604040820540.26807@schroedinger.engr.sgi.com>
Christoph Lameter wrote:
> On Tue, 4 Apr 2006, Nick Piggin wrote:
>
>
>>+ /*
>>+ * PageNoNewRefs is set in order to prevent new references to the
>>+ * page (eg. before it gets removed from pagecache). Wait until it
>>+ * becomes clear (and checks below will ensure we still have the
>>+ * correct one).
>>+ */
>>+ while (unlikely(PageNoNewRefs(page)))
>>+ cpu_relax();
>
>
> That part looks suspiciously like we need some sort of lock here.
>
It's very light-weight now. A lock of course would only be page local,
so it wouldn't really harm scalability, however it would slow down the
single threaded case. At the moment, single threaded performance of
find_get_page is anywhere from about 15-100% faster than before the
lockless patches.
I don't see why you think there needs to be a lock? Before the write
side clears PageNoNewRefs, they will have moved 'page' out of pagecache,
so when this loop breaks, the subsequent test will fail and this
function will be repeated.
--
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
^ permalink raw reply
* No CONFIG_DEBUG_INFO on x86_64 Xen; why not?
From: Stephen C. Tweedie @ 2006-04-06 0:17 UTC (permalink / raw)
To: xen-devel
Hi all,
lib/Kconfig.debug contains:
config DEBUG_INFO
bool "Compile the kernel with debug info"
depends on DEBUG_KERNEL && !X86_64_XEN
Does anyone recall why this was disabled for X86_64_XEN?
hg annotate xen-unstable.hg leads to the cset description
Update linux sparse tree to subarch layout.
and linux-2.6-merge.hg shows
various fixup in Kconfig to handle Xen
neither of which are particularly helpful. I've just completed a build
with this change reverted and -g enabled and it built fine (although I'm
remote from the box right now so can't boot it yet); and gdb on the
vmlinux shows symbolic info is present.
Anyone remember, or object to reenabling CONFIG_DEBUG_INFO?
Thanks,
Stephen
^ permalink raw reply
* blame not working well?
From: Junio C Hamano @ 2006-04-06 0:11 UTC (permalink / raw)
To: Fredrik Kuivinen; +Cc: git
I was having fun updating blame.c to use the built-in xdiff
instead of spawning and reading from external GNU diff (it is
currently in "next" branch). It seems to pass the trivial
testsuite case but I noticed for example annotating Makefile,
sha1_name.c, or blame.c in git.git repository seems to show
quite bogus annotation. One extreme case is the Makefile; for
all but one line is blamed for the very initial commit made by
Linus X-<. One good news for me is that the version before this
change has the same breakage. One bad news is this seems to
have been broken for some time.
Bisecting indicates 2a0925be3512451834ec9a3e023f4cff23c1cfb7 is
the first bad commit, but I do not see how the change can break
it. I'll continue digging it, but if you have a chance, could
you take a look, too?
^ permalink raw reply
* Re: [fuse-devel] Re: [parisc-linux] [PATCH] Fixs to work on ARM and PARIC platforms.
From: James Bottomley @ 2006-04-06 0:07 UTC (permalink / raw)
To: Miklos Szeredi; +Cc: fuse-devel, parisc-linux
In-Reply-To: <E1FQo4N-000159-00@dorka.pomaz.szeredi.hu>
On Tue, 2006-04-04 at 18:07 +0200, Miklos Szeredi wrote:
> > No, because if you look at the implementation, you'll see that
> > copy_to/from_user() copy straight into the user view (i.e. via the user
> > cache lines).
>
> Yes. But how will this ensure that the above problems (dirty/clean
> cache lines in aliases) won't cause any problems?
It doesn't. The object is to write into the user cache as though the
actual user process had done it (i.e. disregarding all aliasing).
> What is the difference between
>
> - updating cached data through the user view
>
> - flushing the cache lines for the user view, then updating data
> through the kernel view and finally flushing the cache lines for
> the kernel view
In the former, the data usually ends up in dirty user cache lines. In
the latter the cache lines are entirely clean after the procedure.
> I don't see in either case that the cache lines for any other aliases
> are touched. Is there still some difference in behavior?
Yes, in the former, only a single alias (the user view) is affected. In
the latter, all views are accounted for.
James
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
^ permalink raw reply
* Re: [parisc-linux] Strange newest LAB msg?
From: James Bottomley @ 2006-04-06 0:02 UTC (permalink / raw)
To: Helge Deller; +Cc: parisc-linux, Grant Grundler
In-Reply-To: <200604050850.06058.deller@gmx.de>
On Wed, 2006-04-05 at 08:50 +0200, Helge Deller wrote:
> > I suggest that on 32 bits, we really shouldn't alter the current scheme
> > (i.e. keep the separated I/O and memory mappings). On 64 bits, we
> > could allocate a far different VM range to vmalloc (somewhere up beyond
> > the maximum possible physical memory) and thus make it far bigger, which
> > would allow us to keep a mapped ioremap implementation.
>
> Might work, although it makes the sources pretty ugly again.
> The biggest problem with 64bit kernel and seperated I/O and memory mappings was, that you wasn't able to export I/O-memory via mmap() to 32bit userspace applications.
> Biggest affected application was X11, who tried to mmap() the 64bit f-space region of the graphics RAM into 32bit-userspace.
> This might maybe be maybe solveable with your proposal, since e.g. X11 on 32bit Kernel worked already somehow and 64bit could be solved with the mapping.
I suspected it might be something like this. Actually, ioremap looks to
be the wrong interface to fix this. The problem seems to be that
fb_mmap() in drivers/video/fb.c has a special fall through case if the
driver doesn't implement fb_mmap() which tries to insert page mappings
for the I/O memory region into the users space, this fall through
doesn't seem to be correct for 64 bits (or at least for 32 bit processes
running on a 64 bit kernel) ... I think if we fix this, we'll get
graphics on 64 bit without the need to remap I/O memory.
James
^ permalink raw reply
* A patch for test_overcommit module
From: Hideo AOKI @ 2006-04-05 23:52 UTC (permalink / raw)
To: akpm; +Cc: linux-kernel, linux-mm
In-Reply-To: <4434570F.9030507@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 161 bytes --]
The test module needs this kernel patch.
I don't intend to propose to apply this patch to kernel tree.
---
Hideo Aoki, Hitachi Computer Products (America) Inc.
[-- Attachment #2: mm-debug.patch --]
[-- Type: text/x-patch, Size: 2086 bytes --]
This patch exports the symbols which a kernel module has to refer.
include/linux/mman.h | 1 +
mm/page_alloc.c | 1 +
mm/slab.c | 1 +
mm/swap.c | 1 +
4 files changed, 4 insertions(+)
diff -pruN linux-2.6.17-rc1-mm1/include/linux/mman.h linux-2.6.17-rc1-mm1-test1/include/linux/mman.h
--- linux-2.6.17-rc1-mm1/include/linux/mman.h 2006-03-24 12:40:19.000000000 -0500
+++ linux-2.6.17-rc1-mm1-test1/include/linux/mman.h 2006-04-04 12:53:52.000000000 -0400
@@ -24,6 +24,7 @@ static inline void vm_acct_memory(long p
{
atomic_add(pages, &vm_committed_space);
}
+EXPORT_SYMBOL(vm_acct_memory);
#endif
static inline void vm_unacct_memory(long pages)
diff -pruN linux-2.6.17-rc1-mm1/mm/page_alloc.c linux-2.6.17-rc1-mm1-test1/mm/page_alloc.c
--- linux-2.6.17-rc1-mm1/mm/page_alloc.c 2006-04-04 10:43:57.000000000 -0400
+++ linux-2.6.17-rc1-mm1-test1/mm/page_alloc.c 2006-04-04 12:53:52.000000000 -0400
@@ -52,6 +52,7 @@ EXPORT_SYMBOL(node_possible_map);
unsigned long totalram_pages __read_mostly;
unsigned long totalhigh_pages __read_mostly;
long nr_swap_pages;
+EXPORT_SYMBOL(nr_swap_pages);
int percpu_pagelist_fraction;
static void __free_pages_ok(struct page *page, unsigned int order);
diff -pruN linux-2.6.17-rc1-mm1/mm/slab.c linux-2.6.17-rc1-mm1-test1/mm/slab.c
--- linux-2.6.17-rc1-mm1/mm/slab.c 2006-04-04 10:43:57.000000000 -0400
+++ linux-2.6.17-rc1-mm1-test1/mm/slab.c 2006-04-04 12:53:52.000000000 -0400
@@ -692,6 +692,7 @@ static struct list_head cache_chain;
* SLAB_RECLAIM_ACCOUNT turns this on per-slab
*/
atomic_t slab_reclaim_pages;
+EXPORT_SYMBOL(slab_reclaim_pages);
/*
* chicken and egg problem: delay the per-cpu array allocation
diff -pruN linux-2.6.17-rc1-mm1/mm/swap.c linux-2.6.17-rc1-mm1-test1/mm/swap.c
--- linux-2.6.17-rc1-mm1/mm/swap.c 2006-04-04 10:43:57.000000000 -0400
+++ linux-2.6.17-rc1-mm1-test1/mm/swap.c 2006-04-04 12:53:52.000000000 -0400
@@ -499,6 +499,7 @@ void vm_acct_memory(long pages)
}
preempt_enable();
}
+EXPORT_SYMBOL(vm_acct_memory);
#ifdef CONFIG_HOTPLUG_CPU
^ permalink raw reply
* A test kernel module of OVERCOMMIT_GUESS
From: Hideo AOKI @ 2006-04-05 23:51 UTC (permalink / raw)
To: akpm; +Cc: linux-kernel, linux-mm
In-Reply-To: <4434570F.9030507@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 978 bytes --]
Hello Andrew,
This is a kernel module patch which I developed to test my patches.
The module makes a kind of memory pressure situation. After that, the
module tests if the OVERCOMMIT_GUESS detects overcommit.
The module has "mode" option. If you specify "mode=1", the module
tries to allocate pages in the test phase.
Here is the test result when I did "mode=1" test on my machine.
* 2.6.17-rc1-mm1
kernel: Test MAY be <failed>.
kernel: allocation failed: out of vmalloc space - use vmalloc=<size> to increase size.
kernel: allocation failed: out of vmalloc space - use vmalloc=<size> to increase size.
kernel: Test SURELY was <FAILED>.
* 2.6.17-rc1-mm1 + my patches
kernel: Test was <PASSED>.
Unfortunately, this kernel module needs another kernel patch.
I will send it in later e-mail.
Please note that I don't intend to propose to apply the module to
kernel tree.
Best regards,
Hideo Aoki
---
Hideo Aoki, Hitachi Computer Products (America) Inc.
[-- Attachment #2: mm-test_overcommit.patch --]
[-- Type: text/x-patch, Size: 14341 bytes --]
A kernel module to test OVERCOMMIT_GUESS.
lib/Kconfig.debug | 11 +
mm/Makefile | 1
mm/test_overcommit.c | 515 +++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 527 insertions(+)
diff -purN linux-2.6.17-rc1-mm1/lib/Kconfig.debug linux-2.6.17-rc1-mm1-test1/lib/Kconfig.debug
--- linux-2.6.17-rc1-mm1/lib/Kconfig.debug 2006-04-04 10:43:57.000000000 -0400
+++ linux-2.6.17-rc1-mm1-test1/lib/Kconfig.debug 2006-04-04 16:30:31.000000000 -0400
@@ -282,6 +282,17 @@ config RCU_TORTURE_TEST
Say M if you want the RCU torture tests to build as a module.
Say N if you are unsure.
+config OVERCOMMIT_GUESS_TEST
+ tristate "An overcommit guess testing module"
+ depends on DEBUG_KERNEL && X86
+ default n
+ help
+ This option provides a kernel module that can test OVERCOMMIT_GUESS
+ in __vm_enough_memory().
+
+ You should say N or M here. Say M if you want to build the module.
+ Say N if you are unsure.
+
config WANT_EXTRA_DEBUG_INFORMATION
bool
select DEBUG_INFO
diff -purN linux-2.6.17-rc1-mm1/mm/Makefile linux-2.6.17-rc1-mm1-test1/mm/Makefile
--- linux-2.6.17-rc1-mm1/mm/Makefile 2006-04-04 10:43:57.000000000 -0400
+++ linux-2.6.17-rc1-mm1-test1/mm/Makefile 2006-04-04 16:23:09.000000000 -0400
@@ -24,4 +24,5 @@ obj-$(CONFIG_SLAB) += slab.o
obj-$(CONFIG_MEMORY_HOTPLUG) += memory_hotplug.o
obj-$(CONFIG_FS_XIP) += filemap_xip.o
obj-$(CONFIG_MIGRATION) += migrate.o
+obj-$(CONFIG_OVERCOMMIT_GUESS_TEST) += test_overcommit.o
diff -purN linux-2.6.17-rc1-mm1/mm/test_overcommit.c linux-2.6.17-rc1-mm1-test1/mm/test_overcommit.c
--- linux-2.6.17-rc1-mm1/mm/test_overcommit.c 1969-12-31 19:00:00.000000000 -0500
+++ linux-2.6.17-rc1-mm1-test1/mm/test_overcommit.c 2006-04-04 16:23:09.000000000 -0400
@@ -0,0 +1,515 @@
+/*
+ * A kernel module for testing OVERCOMMIT_GUESS. ver 0.0.1
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
+ * MA 02110-1301, USA.
+ */
+/*
+ * This kernel module tests the later part of OVERCOMMIT_GUESS algorithm.
+ * To test the algorithm, the module makes a kind of memory pressure situation.
+ * After that, the module tests if the algorithm detects overcommit.
+ *
+ * You can specify by "mode" module option if the module actually tries to
+ * allocate pages in the situation.
+ *
+ * You have to apply a kernel patch to use the module currently, since the
+ * module needs to refer some internal symbols for testing.
+ *
+ * The module was tested on only i386 SMP machines.
+ */
+#include <linux/module.h> /* Needed by all modules */
+#include <linux/moduleparam.h>
+#include <linux/kernel.h> /* Needed for KERN_INFO */
+
+#include <linux/gfp.h>
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/pagemap.h>
+#include <linux/slab.h>
+#include <linux/swap.h>
+#include <linux/vmalloc.h>
+
+#include <asm/kmap_types.h>
+#include <asm/system.h>
+#include <linux/blkdev.h>
+#include <linux/string.h>
+
+/*
+ * Get rid of taint message by declaring code as GPL v2.
+ */
+MODULE_LICENSE("GPL v2");
+
+/*
+ * Module documentation
+ */
+MODULE_AUTHOR("Hideo Aoki");
+MODULE_DESCRIPTION("test module of overcommit guess");
+
+/*
+ * Module parameter(s)
+ */
+enum { MODE_DEFALUT = 0, MODE_DOLASTALLOC = 1 };
+static int mode = MODE_DEFALUT;
+
+module_param(mode, int, S_IRUSR);
+MODULE_PARM_DESC(mode, "test mode. If mode is 1, the module executes actual page allocation in concrete test.");
+
+/*
+ * declarations
+ */
+#define MOD_MM 1 /* modify kernel */
+#define DEF_PRT_LV KERN_DEBUG /* default print level in this mod. */
+
+/*
+ * Page managemnt: We manage allocated pages. When module is unloaded,
+ * the module releases them.
+ */
+struct page_mng_t {
+ struct page *listp; /* list of allocated pages */
+ int zone; /* which zone manage */
+};
+
+/* We use head of allocated pages for managing page list */
+struct mypage_t {
+ struct page *next_page;
+ int order; /* page order */
+};
+
+
+enum {
+ NORMAL, HIGH
+};
+
+static struct page_mng_t low_pg; /* LOWMEM pages management */
+static struct page_mng_t high_pg;/* HIGHMEM pages management */
+static unsigned long committed_pages;/* number of pages committed in the mod.*/
+
+void prepare_test_env(void);
+void final_test(int mode);
+int commit_pages(unsigned long target, struct page_mng_t *mng, int gfp,
+ char *msg);
+int try_to_commit_pages(int order, int gfp, struct page_mng_t *mng,
+ unsigned long *n_alloced);
+int alloc_last_pages(unsigned long npage);
+
+/* allocated page management */
+int page_mng_init(struct page_mng_t *mng, int gfp);
+void page_mng_add(struct page_mng_t *mng, struct page *newp, int order);
+void page_mng_freeall(struct page_mng_t *mng);
+
+/* zone */
+struct zone *get_zone(char *name);
+void print_zone_info(struct zone *zone, char *msg);
+
+
+/*
+ * initialize module and invoke test functions
+ */
+int init_module(void)
+{
+ int ret;
+
+ printk(DEF_PRT_LV "Test module was loaded. <mode %d>\n", mode);
+
+ if (mode != MODE_DEFALUT && mode != MODE_DOLASTALLOC) {
+ printk(KERN_ERR "invalid mode. <mode %d>\n", mode);
+ return 1;
+ }
+
+ /*
+ * making a memory pressure situation
+ */
+ printk(DEF_PRT_LV "init ...");
+ committed_pages = 0;
+ ret = page_mng_init(&low_pg, GFP_KERNEL);
+ if (ret == 1) {
+ printk(KERN_ERR "failed to init lowmem mng\n");
+ return 1;
+ }
+ ret = page_mng_init(&high_pg, GFP_HIGHUSER);
+ if (ret == 1) {
+ printk(KERN_ERR "failed to init highmem mng\n");
+ return 1;
+ }
+ printk(DEF_PRT_LV "done\n");
+
+ prepare_test_env();
+
+ /*
+ * concrete test
+ */
+ printk(DEF_PRT_LV "concrete test ...\n");
+ final_test(mode);
+ printk(DEF_PRT_LV "concrete test ...done.\n");
+
+ /*
+ * A non 0 return means init_module failed; module can't be loaded.
+ */
+ return 0;
+}
+
+
+/*
+ * destructor of module
+ */
+void cleanup_module(void)
+{
+ printk(DEF_PRT_LV "Unloading module ...\n");
+ page_mng_freeall(&low_pg);
+ page_mng_freeall(&high_pg);
+ vm_unacct_memory(committed_pages);
+}
+
+
+/*
+ * To prepare test environment, this function repeat to allocate pages
+ * in ZONE_HIGHMEM and ZONE_NORMAL until the number of free pages is
+ * pages_high in the zone.
+ */
+void prepare_test_env(void)
+{
+ struct zone *high_zone; /* high mem zone */
+ struct zone *normal_zone; /* normal zone */
+ int i;
+ int ret;
+ unsigned long target;
+
+ high_zone = get_zone("HighMem");
+ if (high_zone == NULL) {
+ printk(KERN_ERR "fail to get higmem zone\n");
+ return ;
+ }
+ for (i = 0; i < 100; i++) {
+ if (high_zone->free_pages <= high_zone->pages_high) {
+ printk(DEF_PRT_LV "already satisfied\n");
+ break;
+ }
+
+ spin_lock_irq(&high_zone->lru_lock);
+ target = high_zone->free_pages - high_zone->pages_high;
+ spin_unlock_irq(&high_zone->lru_lock);
+ print_zone_info(high_zone, "HIGH");
+ ret = commit_pages(target, &high_pg, GFP_HIGHUSER, "HighMem");
+ if (ret < 0) {
+ printk(KERN_ERR "error high %i\n", i);
+ goto error;
+ }
+
+ print_zone_info(high_zone, "HIGH");
+ /* printk(DEF_PRT_LV "%i\n", i); */
+ blk_congestion_wait(WRITE, HZ/2);
+ }
+
+ normal_zone = get_zone("Normal");
+ if (high_zone == NULL) {
+ printk(KERN_ERR "fail to get normal zone\n");
+ return ;
+ }
+ for (i = 0; i < 100; i++) {
+ if (normal_zone->free_pages <= normal_zone->pages_high) {
+ printk(DEF_PRT_LV "already satisfied\n");
+ break;
+ }
+
+ spin_lock_irq(&normal_zone->lru_lock);
+ target = normal_zone->free_pages - normal_zone->pages_high;
+ spin_unlock_irq(&normal_zone->lru_lock);
+ print_zone_info(normal_zone, "NORMAL");
+ ret = commit_pages(target, &low_pg, GFP_KERNEL, "Normal");
+ if (ret < 0) {
+ printk(KERN_ERR "error %i\n", i);
+ goto error;
+ }
+
+ print_zone_info(normal_zone, "NORMAL");
+ /* printk(DEF_PRT_LV "%i\n", i); */
+ blk_congestion_wait(WRITE, HZ/2);
+ }
+
+error:
+ return;
+}
+
+/*
+ * main test function
+ */
+void final_test(int mode)
+{
+ int ret;
+ unsigned long n;
+ unsigned long nbuffer;
+ unsigned long ncache;
+ unsigned long nmargin;
+#if MOD_MM
+ unsigned long nslabrec;
+ unsigned long nswap;
+#endif
+ struct sysinfo info;
+
+
+ for (nmargin = 1; nmargin < 100000; nmargin *= 10) {
+ si_meminfo(&info);
+ nbuffer = info.bufferram;
+ ncache = get_page_cache_size();
+#if MOD_MM
+ nslabrec = atomic_read(&slab_reclaim_pages);
+ nswap = nr_swap_pages;
+ n = ncache + nslabrec + nmargin + nswap;
+ printk(DEF_PRT_LV "<buf %lu><cache %lu><slab reclaim %lu><swap %lu> <+ %lu> <target %lu>\n",
+ nbuffer, ncache, nslabrec, nswap, nmargin, n);
+#else
+ n = ncache + nmargin;
+ printk(DEF_PRT_LV "<buf %lu> <cache %lu> <+ %lu> <target %lu>\n",
+ nbuffer, ncache, nmargin, n);
+#endif
+
+ ret = __vm_enough_memory(n, 0);
+ if (ret != 0) {
+ printk(KERN_ERR "Test was <PASSED>.\n");
+ break ;
+ }
+
+ /* unexpected result */
+ committed_pages += n;
+ if (mode == MODE_DOLASTALLOC) {
+ printk(KERN_ERR "Test MAY be <failed>.\n");
+ ret = alloc_last_pages(n);
+ if (ret == 0) {
+ printk(KERN_ERR "Test modeule has problem\n");
+ } else {
+ printk(KERN_ERR "Test SURELY was <FAILED>.\n");
+ break ;
+ }
+ } else {
+ printk(KERN_ERR "Test was <FAILED>\n");
+ }
+ }
+}
+
+int commit_pages(unsigned long target, struct page_mng_t *mng, int gfp,
+ char *msg)
+{
+ int ret;
+ unsigned long total;
+ unsigned long npage;
+
+ if (target == 0 || mng == NULL)
+ goto error;
+
+ /*
+ * try to commit anonymous pages
+ */
+ total = 0;
+ while (total < target) {
+ ret = try_to_commit_pages(0, gfp, mng, &npage);
+ if (ret == 1) {
+ printk(DEF_PRT_LV "%s test stoped. <target %lu>\n",
+ msg, target);
+ return 1;
+ } else if (ret == 0) {
+ total += npage;
+ /* printk(KERN_ERR "p"); */
+ } else if (ret == -1) {
+ printk(KERN_ERR "error %s overcommit.\n", msg);
+ goto error;
+ } else {
+ printk(KERN_ERR "error %s test environment.\n", msg);
+ goto error;
+ }
+ }
+ printk(DEF_PRT_LV "%s <target %lu>, ", msg, target);
+
+ return 0;
+
+ error:
+ return -1;
+}
+
+/*
+ * ret: 1; success (detected overcommit)
+ * 0: success
+ * -1: error (overcommiet was not detected)
+ */
+int try_to_commit_pages(int order, int gfp, struct page_mng_t *mng,
+ unsigned long *n_alloced)
+{
+ int ret;
+ long n_pages;
+ struct page *p;
+
+ n_pages = 1L << order;
+ /* printk(KERN_ERR "<order %d>, <pages %ld>\n,", order, n_pages); */
+
+ *n_alloced = 0;
+
+ ret = __vm_enough_memory(n_pages, 0);
+ if (ret != 0) {
+ printk(KERN_ERR "<order %d>, <pages %ld> ", order, n_pages);
+ printk(KERN_ERR "overcommit was detected.\n");
+ return 1;
+ }
+ committed_pages += n_pages;
+
+ p = alloc_pages(gfp, order);
+ if (p == NULL) {
+ /* error */
+ printk(KERN_ERR "<order %d>, <pages %ld> ", order, n_pages);
+ printk(KERN_ERR "allocation failed\n");
+ return -1;
+ } else {
+ page_mng_add(mng, p, order);
+ *n_alloced += n_pages;
+ /*
+ * printk(KERN_ERR "<order %d>, <pages %ld> <alloced %lu> ",
+ * order, n_pages, *n_alloced);
+ * printk(KERN_ERR " succeed\n");
+ */
+ return 0;
+ }
+}
+
+/*
+ * 0: success, 1: failure
+ */
+int alloc_last_pages(unsigned long npage)
+{
+ int i;
+ int ret;
+ void *mem;
+ struct mypage2_t {
+ struct mypage2_t *next;
+ } *endp, *listp, *p, *nextp;
+
+ /*
+ * allocation scenario 1
+ */
+ mem = vmalloc(npage * PAGE_SIZE);
+ if (mem != NULL) {
+ printk(KERN_ERR "TEST MODULE HAS PROBLEMS.\n");
+ vfree(mem);
+ return 0;
+ }
+
+ /*
+ * allocation scenario 2
+ */
+ for (listp = endp = NULL, i = 0; i < npage; i++) {
+ p = (struct mypage2_t *)vmalloc(PAGE_SIZE);
+ if (p == NULL) {
+ ret = 1;
+ goto release;
+ }
+
+ p->next = NULL;
+ if (listp == NULL)
+ listp = p;
+ else
+ endp->next = p;
+
+ endp = p;
+ }
+
+ printk(KERN_ERR "TEST MODULE HAS PROBLEMS.\n");
+ ret = 0;
+
+release:
+ for ( ; listp != NULL; listp = nextp) {
+ nextp = listp->next;
+ vfree(listp);
+ }
+
+ return ret;
+}
+
+/*
+ * ret: 1: error, 0: success
+ */
+int page_mng_init(struct page_mng_t *mng, int gfp)
+{
+ mng->listp = NULL;
+ if (gfp == GFP_HIGHUSER)
+ mng->zone = HIGH;
+ else if (gfp == GFP_KERNEL || gfp == GFP_USER)
+ mng->zone = NORMAL;
+ else
+ return 1;
+
+ return 0;
+}
+
+void page_mng_add(struct page_mng_t *mng, struct page *newp, int order)
+{
+ struct mypage_t *p;
+
+ if (mng->zone != NORMAL && mng->zone != HIGH) {
+ printk(KERN_ERR "PAGE_MNG: ERROR \n");
+ return ;
+ }
+
+ p = (struct mypage_t *)kmap_atomic(newp, KM_TYPE_NR);
+ p->order = order;
+ p->next_page = mng->listp;
+ mng->listp = newp;
+
+ kunmap_atomic((void *)p, KM_TYPE_NR);
+}
+
+void page_mng_freeall(struct page_mng_t *mng)
+{
+ int order;
+ struct page *next;
+ struct mypage_t *p;
+
+ for ( ; mng->listp != NULL; mng->listp = next) {
+ p = (struct mypage_t *)kmap_atomic(mng->listp, KM_TYPE_NR);
+ next = p->next_page;
+ order = p->order;
+ kunmap_atomic((void *)p, KM_TYPE_NR);
+ __free_pages(mng->listp, order);
+ }
+}
+
+struct zone *get_zone(char *name)
+{
+ struct zone *zone;
+ for_each_zone(zone) {
+ if (strcmp(zone->name, name) == 0)
+ return zone;
+ }
+
+ return NULL;
+}
+
+void print_zone_info(struct zone *zone, char *msg)
+{
+ unsigned long free_pages;
+ unsigned long nr_active;
+ unsigned long nr_inactive;
+ unsigned long present_pages;
+
+ spin_lock_irq(&zone->lru_lock);
+ free_pages = zone->free_pages;
+ nr_active = zone->nr_active;
+ nr_inactive = zone->nr_inactive;
+ present_pages = zone->present_pages;
+ spin_unlock_irq(&zone->lru_lock);
+
+ printk(DEF_PRT_LV "\n%s: <active %lu><inactive %lu><free %lu><sum %lu><present %lu>\n",
+ msg,
+ nr_active,
+ nr_inactive,
+ free_pages,
+ nr_active + nr_inactive + free_pages,
+ present_pages);
+}
^ permalink raw reply
* Re: [patch 2/3] mm: speculative get_page
From: Nick Piggin @ 2006-04-05 0:22 UTC (permalink / raw)
To: Christoph Lameter
Cc: Nick Piggin, Andrew Morton, Linux Kernel, Linux Memory Management
In-Reply-To: <Pine.LNX.4.64.0604040814140.26807@schroedinger.engr.sgi.com>
Christoph Lameter wrote:
> Looks like the NoNewRefs flag is mostly ==
> spin_is_locked(mapping->tree_lock)? Would it not be better to check the
> tree_lock?
>
Well there are other uses for the tree_lock (eg. tag operations)
which do not need the "no new references" guarantee.
>
>
>>--- linux-2.6.orig/mm/migrate.c
>>+++ linux-2.6/mm/migrate.c
>>
>>+ SetPageNoNewRefs(page);
>> write_lock_irq(&mapping->tree_lock);
>
>
> A dream come true! If this is really working as it sounds then we can
> move the SetPageNoNewRefs up and avoid the final check under
> mapping->tree_lock. Then keep SetPageNoNewRefs until the page has been
> copied. It would basically play the same role as locking the page.
>
Yes we could do that but at this stage I wouldn't like to seperate
SetPageNoNewRefs from tree_lock, as it is replacing a traditional
guarantee that tree_lock no longer provides.
--
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
^ permalink raw reply
* [patch 3/3] mm: An enhancement of OVERCOMMIT_GUESS
From: Hideo AOKI @ 2006-04-05 23:48 UTC (permalink / raw)
To: akpm; +Cc: linux-kernel, linux-mm
[-- Attachment #1: Type: text/plain, Size: 182 bytes --]
There is a copy of __vm_enough_memory() in mm/nommu.c. I believe that
this enhancement is useful for nommu environment too.
---
Hideo Aoki, Hitachi Computer Products (America) Inc.
[-- Attachment #2: mm-consider_rsvpgs-nommu.patch --]
[-- Type: text/x-patch, Size: 1394 bytes --]
This patch is an enhancement of OVERCOMMIT_GUESS algorithm in
__vm_enough_memory() in mm/nommu.c.
When the OVERCOMMIT_GUESS algorithm calculates the number of free
pages, the algorithm subtracts the number of reserved pages from
the result nr_free_pages().
Signed-off-by: Hideo Aoki <haoki@redhat.com>
---
nommu.c | 18 +++++++++++++++---
1 files changed, 15 insertions(+), 3 deletions(-)
diff -purN linux-2.6.17-rc1-mm1/mm/nommu.c linux-2.6.17-rc1-mm1-idea6/mm/nommu.c
--- linux-2.6.17-rc1-mm1/mm/nommu.c 2006-04-04 10:43:30.000000000 -0400
+++ linux-2.6.17-rc1-mm1-idea6/mm/nommu.c 2006-04-04 15:09:24.000000000 -0400
@@ -1147,14 +1147,26 @@ int __vm_enough_memory(long pages, int c
* only call if we're about to fail.
*/
n = nr_free_pages();
+
+ /*
+ * Leave reserved pages. The pages are not for anonymous pages.
+ */
+ if (n <= totalreserve_pages)
+ goto error;
+ else
+ n -= totalreserve_pages;
+
+ /*
+ * Leave the last 3% for root
+ */
if (!cap_sys_admin)
n -= n / 32;
free += n;
if (free > pages)
return 0;
- vm_unacct_memory(pages);
- return -ENOMEM;
+
+ goto error;
}
allowed = totalram_pages * sysctl_overcommit_ratio / 100;
@@ -1175,7 +1187,7 @@ int __vm_enough_memory(long pages, int c
*/
if (atomic_read(&vm_committed_space) < (long)allowed)
return 0;
-
+error:
vm_unacct_memory(pages);
return -ENOMEM;
^ permalink raw reply
* [patch 2/3] mm: An enhancement of OVERCOMMIT_GUESS
From: Hideo AOKI @ 2006-04-05 23:48 UTC (permalink / raw)
To: akpm; +Cc: linux-kernel, linux-mm
[-- Attachment #1: Type: text/plain, Size: 187 bytes --]
This patch is the main part of this enhancement.
Additionally, the patch includes better error handling in __vm_enough_memory().
---
Hideo Aoki, Hitachi Computer Products (America) Inc.
[-- Attachment #2: mm-consider_rsvpgs.patch --]
[-- Type: text/x-patch, Size: 1380 bytes --]
This patch is an enhancement of OVERCOMMIT_GUESS algorithm in
__vm_enough_memory() in mm/mmap.c.
When the OVERCOMMIT_GUESS algorithm calculates the number of free
pages, the algorithm subtracts the number of reserved pages from
the result nr_free_pages().
Signed-off-by: Hideo Aoki <haoki@redhat.com>
---
mmap.c | 18 +++++++++++++++---
1 files changed, 15 insertions(+), 3 deletions(-)
diff -purN linux-2.6.17-rc1-mm1/mm/mmap.c linux-2.6.17-rc1-mm1-idea6/mm/mmap.c
--- linux-2.6.17-rc1-mm1/mm/mmap.c 2006-04-04 10:43:57.000000000 -0400
+++ linux-2.6.17-rc1-mm1-idea6/mm/mmap.c 2006-04-04 14:56:51.000000000 -0400
@@ -121,14 +121,26 @@ int __vm_enough_memory(long pages, int c
* only call if we're about to fail.
*/
n = nr_free_pages();
+
+ /*
+ * Leave reserved pages. The pages are not for anonymous pages.
+ */
+ if (n <= totalreserve_pages)
+ goto error;
+ else
+ n -= totalreserve_pages;
+
+ /*
+ * Leave the last 3% for root
+ */
if (!cap_sys_admin)
n -= n / 32;
free += n;
if (free > pages)
return 0;
- vm_unacct_memory(pages);
- return -ENOMEM;
+
+ goto error;
}
allowed = (totalram_pages - hugetlb_total_pages())
@@ -150,7 +162,7 @@ int __vm_enough_memory(long pages, int c
*/
if (atomic_read(&vm_committed_space) < (long)allowed)
return 0;
-
+error:
vm_unacct_memory(pages);
return -ENOMEM;
^ permalink raw reply
* [patch 1/3] mm: An enhancement of OVERCOMMIT_GUESS
From: Hideo AOKI @ 2006-04-05 23:47 UTC (permalink / raw)
To: akpm; +Cc: linux-kernel, linux-mm
[-- Attachment #1: Type: text/plain, Size: 562 bytes --]
Hello Andrew,
Could you apply my patches to your tree?
These patches are an enhancement of OVERCOMMIT_GUESS algorithm in
__vm_enough_memory(). The detailed description is in attached patch.
Actually, these are the revised patch which I sent to lkml in the last
year.
http://marc.theaimsgroup.com/?l=linux-kernel&m=112993489022427&w=2
I wrote a test kernel module to show the result of the patches.
For your information, I also would like to send the module in later e-mail.
Best regards,
Hideo Aoki
---
Hideo Aoki, Hitachi Computer Products (America) Inc.
[-- Attachment #2: mm-add-totalreserve_pages.patch --]
[-- Type: text/x-patch, Size: 4960 bytes --]
These patches are an enhancement of OVERCOMMIT_GUESS algorithm in
__vm_enough_memory().
- why the kernel needed patching
When the kernel can't allocate anonymous pages in practice, currnet
OVERCOMMIT_GUESS could return success. This implementation might be
the cause of oom kill in memory pressure situation.
If the Linux runs with page reservation features like
/proc/sys/vm/lowmem_reserve_ratio and without swap region, I think
the oom kill occurs easily.
- the overall design approach in the patch
When the OVERCOMMET_GUESS algorithm calculates number of free pages,
the reserved free pages are regarded as non-free pages.
This change helps to avoid the pitfall that the number of free pages
become less than the number which the kernel tries to keep free.
- testing results
I tested the patches using my test kernel module.
If the patches aren't applied to the kernel, __vm_enough_memory()
returns success in the situation but autual page allocation is
failed.
On the other hand, if the patches are applied to the kernel, memory
allocation failure is avoided since __vm_enough_memory() returns
failure in the situation.
I checked that on i386 SMP 16GB memory machine. I haven't tested on
nommu environment currently.
- changelog
v5:
- updated to 2.6.17-rc1-mm1
- did more strict tests.
- added the enhancement to mm/nommu.c too
v4:
- dealing with pages_high as reserved pages
- updated the code for 2.6.14-rc4-mm1
v3 (private):
- enhanced error handling in __vm_enough_memory
- fixed an issue related calculation of totalreserve_pages
v2 (private):
- fixed error handling bug
- updated test results
- updated the code for 2.6.14-rc2-mm2
This patch adds totalreserve_pages for __vm_enough_memory().
Calculate_totalreserve_pages() checks maximum lowmem_reserve pages and
pages_high in each zone. Finally, the function stores the sum of each
zone to totalreserve_pages.
The totalreserve_pages is calculated when the VM is initilized.
And the variable is updated when /proc/sys/vm/lowmem_reserve_raito
or /proc/sys/vm/min_free_kbytes are changed.
Signed-off-by: Hideo Aoki <haoki@redhat.com>
---
include/linux/swap.h | 1 +
mm/page_alloc.c | 39 +++++++++++++++++++++++++++++++++++++++
2 files changed, 40 insertions(+)
diff -purN linux-2.6.17-rc1-mm1/include/linux/swap.h linux-2.6.17-rc1-mm1-idea6/include/linux/swap.h
--- linux-2.6.17-rc1-mm1/include/linux/swap.h 2006-04-04 10:43:57.000000000 -0400
+++ linux-2.6.17-rc1-mm1-idea6/include/linux/swap.h 2006-04-04 15:13:26.000000000 -0400
@@ -155,6 +155,7 @@ extern void swapin_readahead(swp_entry_t
/* linux/mm/page_alloc.c */
extern unsigned long totalram_pages;
extern unsigned long totalhigh_pages;
+extern unsigned long totalreserve_pages;
extern long nr_swap_pages;
extern unsigned int nr_free_pages(void);
extern unsigned int nr_free_pages_pgdat(pg_data_t *pgdat);
diff -purN linux-2.6.17-rc1-mm1/mm/page_alloc.c linux-2.6.17-rc1-mm1-idea6/mm/page_alloc.c
--- linux-2.6.17-rc1-mm1/mm/page_alloc.c 2006-04-04 10:43:57.000000000 -0400
+++ linux-2.6.17-rc1-mm1-idea6/mm/page_alloc.c 2006-04-04 15:13:26.000000000 -0400
@@ -51,6 +51,7 @@ nodemask_t node_possible_map __read_most
EXPORT_SYMBOL(node_possible_map);
unsigned long totalram_pages __read_mostly;
unsigned long totalhigh_pages __read_mostly;
+unsigned long totalreserve_pages __read_mostly;
long nr_swap_pages;
int percpu_pagelist_fraction;
@@ -2548,6 +2549,38 @@ void __init page_alloc_init(void)
}
/*
+ * calculate_totalreserve_pages - called when sysctl_lower_zone_reserve_ratio
+ * or min_free_kbytes changes.
+ */
+static void calculate_totalreserve_pages(void)
+{
+ struct pglist_data *pgdat;
+ unsigned long reserve_pages = 0;
+ int i, j;
+
+ for_each_online_pgdat(pgdat) {
+ for (i = 0; i < MAX_NR_ZONES; i++) {
+ struct zone *zone = pgdat->node_zones + i;
+ unsigned long max = 0;
+
+ /* Find valid and maximum lowmem_reserve in the zone */
+ for (j = i; j < MAX_NR_ZONES; j++) {
+ if (zone->lowmem_reserve[j] > max)
+ max = zone->lowmem_reserve[j];
+ }
+
+ /* we treat pages_high as reserved pages. */
+ max += zone->pages_high;
+
+ if (max > zone->present_pages)
+ max = zone->present_pages;
+ reserve_pages += max;
+ }
+ }
+ totalreserve_pages = reserve_pages;
+}
+
+/*
* setup_per_zone_lowmem_reserve - called whenever
* sysctl_lower_zone_reserve_ratio changes. Ensures that each zone
* has a correct pages reserved value, so an adequate number of
@@ -2578,6 +2611,9 @@ static void setup_per_zone_lowmem_reserv
}
}
}
+
+ /* update totalreserve_pages */
+ calculate_totalreserve_pages();
}
/*
@@ -2632,6 +2668,9 @@ void setup_per_zone_pages_min(void)
zone->pages_high = zone->pages_min + tmp / 2;
spin_unlock_irqrestore(&zone->lru_lock, flags);
}
+
+ /* update totalreserve_pages */
+ calculate_totalreserve_pages();
}
/*
^ permalink raw reply
* Re: [2.6.16 PATCH] Filessytem Events Reporter V2
From: Yi Yang @ 2006-04-05 23:45 UTC (permalink / raw)
To: Carlos Silva; +Cc: LKML, Andrew Morton, Evgeniy Polyakov, Matt Helsley
In-Reply-To: <1144257124.2075.10.camel@localhost>
Carlos Silva wrote:
> On Wed, 2006-04-05 at 21:21 +0800, Yi Yang wrote:
>
>> <snip>
>> +static void cleanup_dead_listener(listener * x)
>> +{
>> + pid_filter * p = NULL, * pq = NULL;
>> + uid_filter * u = NULL, * uq = NULL;
>> + gid_filter * g = NULL, * gq = NULL;
>> +
>> + if (p == NULL)
>> + return;
>> <snip>
>>
>
> I think you ment "if (x == NULL)" here. Otherwise, the condition will always be true.
> btw, I'm not reviewing your code, just stumbled across this while reading it.
>
Yes, it is my mistake, thank you.
^ permalink raw reply
* Re: [2.6.16 PATCH] Filessytem Events Reporter V2
From: Yi Yang @ 2006-04-05 23:44 UTC (permalink / raw)
To: Michael Guo; +Cc: LKML, Andrew Morton, Evgeniy Polyakov, Matt Helsley
In-Reply-To: <1A60C93388AFD3419AEE0E20A116D3201CFDA1@exch2k>
Michael Guo wrote:
> Hi,
> Now, kernel is growing bigger and bigger continuously and performance is becoming slower. So, if possible, please consider to add a
> common and simple interface which is scalable and flexible to satisfy real requirement of users instead of telling users use this or that like Microsoft. In a word, simple makes application programmers happy!
>
>
> Guo
>
Your requirement should be done by a userspace library, in fact, most of
functions provided by kernel are exported
to the final application in this way
^ permalink raw reply
* [lm-sensors] ali1563 problems
From: Jo Lodewyckx @ 2006-04-05 23:37 UTC (permalink / raw)
To: lm-sensors
In-Reply-To: <000001c657ef$bd81d2c0$937ba8c0@genesis>
Hello,
The contents of /proc/ioports:
regawebdev proc # cat ioports
0000-001f : dma1
0020-0021 : pic1
0040-0043 : timer0
0050-0053 : timer1
0060-006f : keyboard
0070-0077 : rtc
0080-008f : dma page reg
00a0-00a1 : pic2
00c0-00df : dma2
00f0-00ff : fpu
01f0-01f7 : ide0
0295-0296 : pnp 00:07
03c0-03df : vga+
03f6-03f6 : ide0
0400-040f : i2c-ali1563
0680-06ff : pnp 00:07
0800-083f : 0000:00:03.1
0800-0803 : PM1a_EVT_BLK
0804-0805 : PM1a_CNT_BLK
0808-080b : PM_TMR
0810-0815 : ACPI CPU throttle
0818-0827 : GPE0_BLK
0830-0830 : PM2_CNT_BLK
0cf8-0cff : PCI conf1
e000-efff : PCI Bus #02
ec00-ec3f : 0000:02:06.0
ec00-ec3f : e1000
ff00-ff0f : 0000:00:0e.0
ff00-ff07 : ide0
ff08-ff0f : ide1
-----Original Message-----
From: Jean Delvare [mailto:khali at linux-fr.org]
Sent: woensdag 5 april 2006 11:03
To: Jo Lodewyckx
Cc: LM Sensors
Subject: Re: [lm-sensors] ali1563 problems
> > Do you want to scan the ISA bus? (YES/no):
> >
> > Probing for `Winbond W83627HF'
> > Trying address 0x0290... Success!
> > (confidence 8, driver `w83781d')
> >
> > Some Super I/O chips may also contain sensors. Super I/O probes are
> > typically a bit more dangerous, as we have to write to I/O ports to do
> > this. This is usually safe though.
> >
> > Do you want to scan for Super I/O sensors? (YES/no):
> > Probing for `Winbond W83627HF Super IO Sensors'
> > Success... found at address 0x0290/
>
> Please can you provide dump of:
>
> isadump 0x295 0x296
And even before that, the contents of /proc/ioports
Kernel 2.6.14 was still affected by the PNP ACPI vs w83627hf issue, we
fixed it only in 2.6.15.
(See http://bugzilla.kernel.org/show_bug.cgi?id@14 for details.)
--
Jean Delvare
^ permalink raw reply
* [lm-sensors] ali1563 problems
From: Jo Lodewyckx @ 2006-04-05 23:35 UTC (permalink / raw)
To: lm-sensors
In-Reply-To: <000001c657ef$bd81d2c0$937ba8c0@genesis>
Hi Rudolf,
Here's the isadump:
regawebdev jo # isadump 0x295 0x296
WARNING! Running this program can cause system crashes, data loss and worse!
I will probe address register 0x295 and data register 0x296.
Continue? [Y/n]
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
10: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
20: 5b 5c cd bb c7 c7 c8 15 38 35 ff ff 00 ff 00 ed
30: b0 52 3d e1 4b 50 8c 42 79 33 78 4e 3c ff 00 00
40: 01 08 06 00 00 00 00 ff 2d 03 01 44 08 15 80 5c
50: ff ff 00 ff ff ff 00 80 21 70 ff ff 11 00 ff 05
60: 5b 5c cd bb c7 c7 c8 15 38 35 ff ff 00 ff 00 ed
70: b0 52 3d e1 4b 50 8c 42 79 33 78 4e 3c ff 00 00
80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
90: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
a0: 5b 5c cd bb c7 c7 c8 15 38 35 ff ff 00 ff 00 ed
b0: b0 52 3d e1 4b 50 8c 42 79 33 78 4e 3c ff 00 00
c0: 01 00 00 00 00 00 00 ff 2d 03 01 44 08 15 80 5c
d0: ff ff 00 ff ff ff 00 80 21 70 ff ff 11 00 ff 05
e0: 5b 5c cd bb c7 c7 c8 15 38 35 ff ff 00 ff 00 ed
f0: b0 52 3d e1 4b 50 8c 42 79 33 78 4e 3c ff 00 00
Regards,
Jo
-----Original Message-----
From: Rudolf Marek [mailto:r.marek at sh.cvut.cz]
Sent: woensdag 5 april 2006 9:51
To: Jo Lodewyckx
Cc: lm-sensors at lm-sensors.org
Subject: Re: [lm-sensors] ali1563 problems
> Do you want to scan the ISA bus? (YES/no):
>
> Probing for `Winbond W83627HF'
> Trying address 0x0290... Success!
> (confidence 8, driver `w83781d')
>
> Some Super I/O chips may also contain sensors. Super I/O probes are
> typically a bit more dangerous, as we have to write to I/O ports to do
> this. This is usually safe though.
>
> Do you want to scan for Super I/O sensors? (YES/no):
> Probing for `Winbond W83627HF Super IO Sensors'
> Success... found at address 0x0290/
Please can you provide dump of:
isadump 0x295 0x296
Thanks,
Regards
Rudolf
^ permalink raw reply
* RE: PAE dom0 is broken with the default config
From: Ian Pratt @ 2006-04-05 23:33 UTC (permalink / raw)
To: Kamble, Nitin A, Keir.Fraser; +Cc: xen-devel
> I built PAE Xen with the default config, and it is not
> booting. Log is bellow. If I replace the
> linux-defconfig_xen_x86_32 file with the
> linux-defconfig_xen0_x86_32 before building then dom0 is able
> to boot and see more than 4Gig memory. This looks like a dom0
> kernel config issue. Do you care PAE dom0 for 3.0.2 with the
> default config? Is anybody working on this?
This seems pretty unlikely to be a PAE issue -- we do quite extensive
testing.
Are you sure you have the right initrd? Is the same scsi driver being
used as in the -xen0 case?
Thanks,
Ian
>
>
> IP route cache hash table entries: 262144 (order: 8, 1048576 bytes)
>
> TCP established hash table entries: 524288 (order: 10, 4194304 bytes)
>
> TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
>
> TCP: Hash tables configured (established 524288 bind 65536)
>
> TCP reno registered
>
> NET: Registered protocol family 8
>
> NET: Registered protocol family 20
>
> Using IPI No-Shortcut mode
>
> Freeing unused kernel memory: 172k freed
>
> scsi_mod: no version for "struct_module" found: kernel tainted.
>
> SCSI subsystem initialized
>
> ACPI: PCI Interrupt 0000:00:1f.2[A] -> GSI 18 (level, low) -> IRQ 17
>
> ata1: SATA max UDMA/133 cmd 0x0 ctl 0xC402 bmdma 0xBC00 irq 17
>
> ata2: SATA max UDMA/133 cmd 0x0 ctl 0xC002 bmdma 0xBC08 irq 17
>
> ata1: dev 0 ATA-6, max UDMA/133, 312581808 sectors: LBA48
>
> ata1: dev 0 configured for UDMA/133
>
> scsi0 : ata_piix
>
> ATA: abnormal status 0x7F on port 0xC087
>
> scsi1 : ata_piix
>
> Vendor: ATA Model: ST3160023AS Rev: 3.00
>
> Type: Direct-Access ANSI SCSI revision: 05
>
> SCSI device sda: 312581808 512-byte hdwr sectors (160042 MB)
>
> sda: Write Protect is off
>
> SCSI device sda: drive cache: write back
>
> SCSI device sda: 312581808 512-byte hdwr sectors (160042 MB)
>
> sda: Write Protect is off
>
> SCSI device sda: drive cache: write back
>
> sda: unknown partition table
>
> sd 0:0:0:0: Attached scsi disk sda
>
> Kernel panic - not syncing: Attempted to kill init!
>
> (XEN) Domain 0 crashed: rebooting machine in 5 seconds.
>
>
>
>
>
>
>
>
>
>
>
>
^ permalink raw reply
* Re: [PATCH] Fix unneeded rebuilds in drivers/media/video after moving source tree
From: Carl-Daniel Hailfinger @ 2006-04-05 23:32 UTC (permalink / raw)
To: Sam Ravnborg; +Cc: Linux Kernel Mailing List
In-Reply-To: <20060404150233.GA10608@mars.ravnborg.org>
Sam Ravnborg schrieb:
>>This fixes some uneeded rebuilds under drivers/media/video after moving
>>the source tree. The makefiles used $(src) and $(srctree) for include
>>paths, which is unnecessary. Changed to use relative paths.
>>
>>Compile tested, produces byte-identical code to the previous makefiles.
>
>
> Thanks, applied both.
> Did similar changes in bt8xx + some ppc code.
Thanks. Was it intentional to leave the line below in
drivers/media/video/bt8xx/Makefile?
EXTRA_CFLAGS += -I$(src)/..
I think it could be replaced with
EXTRA_CFLAGS += -Idrivers/media/video
but I'm not a kbuild expert.
Regards,
Carl-Daniel
--
http://www.hailfinger.org/
^ permalink raw reply
* Re: Cygwin can't handle huge packfiles?
From: Rutger Nijlunsing @ 2006-04-05 23:27 UTC (permalink / raw)
To: Christopher Faylor; +Cc: Johannes Schindelin, Kees-Jan Dijkzeul, git
In-Reply-To: <20060405210844.GN26780@trixie.casa.cgf.cx>
On Wed, Apr 05, 2006 at 05:08:44PM -0400, Christopher Faylor wrote:
> On Wed, Apr 05, 2006 at 04:14:20PM +0200, Johannes Schindelin wrote:
> >> Inspired by a patch of Alex Riesen (thanks, Alex), I tried to use the
> >> regular mmap for mapping pack files, only to discover that I compile
> >> without defining "NO_MMAP", so I've been using the stock mmap all
> >> along. So now I'm thinking that the cygwin mmap also does a
> >> malloc-and-read, just like git does with NO_MMAP. So I'll continue to
> >> investigate in that direction.
> >
> >I think cygwin's mmap() is based on the Win32 API equivalent, which could
> >mean that it *is* memory mapped, but in a special area (which is smaller
> >than 1.5 gigabyte). In this case, it would make sense to limit the pack
> >size, thereby having several packs, and mmap() them as they are needed.
>
> Yes, cygwin's mmap uses CreateFileMapping and MapViewOfFile. IIRC,
> Windows might have a 2G limitation lurking under the hood somewhere but
> I think that might be tweakable with some registry setting.
Windows places its DLLs criss-cross through the memory space because
every DLL on the system has its own preferred place to be loaded (the
base address). This severely limits the amount of largest contiguous
memory block available, which is needed for one mmap() I think.
Several solutions exist:
- enlarge the address space with the /3GB boot flag in boot.ini
- rebase all DLLs with REBASE.EXE (part of platform sdk) .
Just make them the same and fix them to a low address.
Problem is rebasing system dlls since those are locked by the system.
- at start of program before other DLLs are loaded,
reserve an as large part of the memory as possible with
VirtualAlloc()
--
Rutger Nijlunsing ---------------------------------- eludias ed dse.nl
never attribute to a conspiracy which can be explained by incompetence
----------------------------------------------------------------------
^ permalink raw reply
* [linux-lvm] lvm2 pv failure
From: Uk1ah Sm1th @ 2006-04-05 23:26 UTC (permalink / raw)
To: linux-lvm
Hello-
I had one of the two drives in my LVM2 array fail. It was the first
drive. Is it possiable that any data on the second drive could be
recovered? Or have I lost all my data on both drives?
.Ukiah Smith
--
"Well done is better than well said."
- Benjamin Franklin (1706-1790)
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.