Re: Benchmarking objrmap under memory pressure

Andrea Arcangeli
2004-04-15T13:22:57Z

On Thu, Apr 15, 2004 at 11:21:30AM +0100, Hugh Dickins wrote:
> On Thu, 15 Apr 2004, Andrea Arcangeli wrote:
> > On Wed, Apr 14, 2004 at 06:48:40PM +0100, Hugh Dickins wrote:
> > > This is just your guess at present, isn't it, Andrea? Any evidence?
> >
> > the evidence is pretty obvious, the single fact it's painful to remove
> > the page_table_lock with anonmm around the vma manipulations, and the
> > little benefit that the vma->page_table_lock provides with anonmm is
> > quite a tangible measurements, I'm talking about the 256 ways here, any
> > UP measurements is pretty useless.
>
> Quite possibly. Quite possibly not (anonmm can perfectly well use a
> different lock than the page_table_lock to make find_vma safe against
> try_to_unmap, if page_table_lock too contended, or split into vmas).

"use another global mm wide lock" == less scalable

> A good hypothesis for you to base your design on. But the evidence
> is yet to come. My own bet is that it will make very little difference
> to 256-way performance, whether anonmm or anon_vma: that their issues
> will be with the file-based.

it depends on the workload.

> The worst that will happen with anonmm's mremap move, is that some
> app might go slower and need more swap. Unlikely, but agreed possible.

this is exactly the point and I cannot take that risk in -aa, perod,
taking a dozen more mbytes of ram with some dozen kde task in a desktop
machine would be an huge penalty, the kde people worked hard to make the
library design possible and save ram with anonymous COW memory whenever
possible, I cannot risk to screw their effort, plus I've absolutely no
idea if any big proprietary app may be hurted too, and I cannot take
that risk either, the 12 bytes per vma are nothing compared a potential
invalidation of cow through mremap, plus the anon-vma provided other
advantages as well.

> In your case, some app may actually break (I was going to say
> mysteriously, but that's unfair, ptrace should quickly identify it):
> because of your limitations on anon vma merging, and the way mremap
> is only allowed on a single vma. Again, unlikely, but possible.

any application relaying on the vma merging to make mremap work is
definitely totally broken period, no need to argue further about this
point. the vma merging is a "best effort" provided by the VM, it's
something userspace must not be aware about since it can change over
time, and it's not available at all with mlock and with older 2.4
kernels. so you're definitely wrong claiming I take any risk with
mremap, plus I'm actually _fixing_ mremap by disabling the vma merging
there, and that's the primary reason I disabled the vma merging, the vma
merging of mremap has never been able to retire correctly, so if there
would be an oom failure during a pagetable allocation the vma merging in
2.4 the extension generated by the vma merging wouldn't be retired
correctly. I didn't fix that in 2.4 because it's not possible to
reproduce an oom failure exactly in the move_page_tables and secondly
because it's not exploitable anyways even if it triggers the oom failure
there, unless you also can join this kernel bug with an userspace bug,
so the probability is near zero and even in the extremely unlikely case
it's not exploitable remotely, but with anon-vma I made the probability
zero by disabling the merging in 2.6, until it gets implemented properly
(maybe we should disable it in 2.4 too and go 100% safe, but I didn't
bother to disable it in 2.4).

as for mprotect you're right it can in some extremely unlikely case
merge less, but that's the feature, so the swapping will be more
efficient, and the locking scalable too during paging. It's a trade-off
and I think it makes lots of sense. If an application wants merging it
should create adiacent mappings in the first place. Secondly if an
applications creates an hole inside an anonymous mapping I still
guarantee that the gap-fill will be merged 100%. So I think this is an
unrelistic problem, something I don't need to worry about, unlike the
mremap cow-break that anonmm might introduce in some app.

> I'm sure the right answer to that is to fix mremap to work across vmas:

the limitation on the mremap on a single vma is stupid indeed, it's just
that the implementation was lazy, that needs fixing anyways eventually,
but it's not a problem at all as far as anon-vma is concerned, in no way
an application can relay on the vma merging to do mremap or mlock and
older 2.4 kernels would break too.

> it's unique and wrong to be letting the kernel implementation detail of
> vma percolate up to userspace semantics. But, on the other hand,
> I'm glad of any excuse not to have to go in there and fix it!

I definitely don't need to go in there and fix it either.

> (Of course, your file vma merging will make some mremaps
> possible which were ruled out before: that's nice.)

yes, that's a major benefit, doing vma merging with file mappings is a
lot more important than for anonymous ram, most people only uses
mprotect to switch the write bit on the vma before/after using some
MAP_SHARED segment, if a bug accours while they don't use the mapping
they won't risk to corrupt the data. That's a very common behaviour for
big apps and it has never been possible to merge until now in anon-vma.

But this is pretty orthgonal with the anon-vma vs anonmm comparison, if
you're ok to deal with the anon-vma complexity you can merge this bit on
top of anonmm too, the compexity of doing anon-vma merging is the same
one of doing the inode-vma merging.