Re: [PATCH 2/4] mm: Send a single IPI to TLB flush multiple pages when unmapping

Hugh Dickins
2015-04-15T21:17:04Z

On Wed, 15 Apr 2015, Rik van Riel wrote:
> On 04/15/2015 06:42 AM, Mel Gorman wrote:
> > An IPI is sent to flush remote TLBs when a page is unmapped that was
> > recently accessed by other CPUs. There are many circumstances where this
> > happens but the obvious one is kswapd reclaiming pages belonging to a
> > running process as kswapd and the task are likely running on separate CPUs.
> >
> > On small machines, this is not a significant problem but as machine
> > gets larger with more cores and more memory, the cost of these IPIs can
> > be high. This patch uses a structure similar in principle to a pagevec
> > to collect a list of PFNs and CPUs that require flushing. It then sends
> > one IPI to flush the list of PFNs. A new TLB flush helper is required for
> > this and one is added for x86. Other architectures will need to decide if
> > batching like this is both safe and worth the memory overhead. Specifically
> > the requirement is;
> >
> > If a clean page is unmapped and not immediately flushed, the
> > architecture must guarantee that a write to that page from a CPU
> > with a cached TLB entry will trap a page fault.
> >
> > This is essentially what the kernel already depends on but the window is
> > much larger with this patch applied and is worth highlighting.
>
> This means we already have a (hard to hit?) data corruption
> issue in the kernel. We can lose data if we unmap a writable
> but not dirty pte from a file page, and the task writes before
> we flush the TLB.

I don't think so. IIRC, when the CPU needs to set the dirty bit,
it doesn't just do that in its TLB entry, but has to fetch and update
the actual pte entry - and at that point discovers it's no longer
valid so traps, as Mel says.

(I'm now reading that paragraph differently from when I replied to 4/4.)

Hugh