Re: [RFC] Parallelize IO for e2fsck

Bryan Henderson
2008-01-26T01:56:37Z

>> Incidentally, some context for the AIX approach to the OOM problem: a
>> process may exclude itself from OOM vulnerability altogether. It
places
>> itself in "early allocation" mode, which means at the time it creates
>> virtual memory, it reserves enough backing store for the worst case.
The
>> memory manager does not send such a process the SIGDANGER signal or
>> terminate it when it runs out of paging space. Before c. 2000, this
was
>> the only mode. Now the default is late allocation mode, which is
similar
>> to Linux.
>
>This is an interesting approach. It feels like some programs might be
>interested in choosing this mode instead of risking OOM.

It's the way virtual memory always worked when it was first invented. The
system not only reserved space to back every page of virtual memory; it
assigned the particular blocks for it. Late allocation was a later
innovation, and I believe its main goal was to make it possible to use the
cheaper disk drives for paging instead of drums. Late allocation gives
you better locality on disk, so the seeking doesn't eat you alive (drums
don't seek). Even then, I assume (but am not sure) that the system at
least reserved the space in an account somewhere so at pageout time there
was guaranteed to be a place to which to page out. Overcommitting page
space to save on disk space was a later idea.

I was surprised to see AIX do late allocation by default, because IBM's
traditional style is bulletproof systems. A system where a process can be
killed at unpredictable times because of resource demands of unrelated
processes doesn't really fit that style.

It's really a fairly unusual application that benefits from late
allocation: one that creates a lot more virtual memory than it ever
touches. For example, a sparse array. Or am I missing something?

--
Bryan Henderson IBM Almaden Research Center
San Jose CA Filesystems