Re: [PATCH v3 21/26] mm/page_alloc: implement FREETYPE_UNMAPPED allocations

[Yosry Ahmed <yosry@kernel.org>]

> @@ -3400,6 +3426,127 @@ static inline void zone_statistics(struct zone *preferred_zone, struct zone *z,
>  #endif
>  }
>  
> +#ifdef CONFIG_PAGE_ALLOC_UNMAPPED
> +/* Try to allocate a page by mapping/unmapping a block from the direct map. */
> +static inline struct page *
> +__rmqueue_direct_map(struct zone *zone, unsigned int request_order,
> +		     unsigned int alloc_flags, freetype_t freetype)
> +{
> +	unsigned int ft_flags_other = freetype_flags(freetype) ^ FREETYPE_UNMAPPED;
> +	freetype_t ft_other = migrate_to_freetype(free_to_migratetype(freetype),
> +						  ft_flags_other);
> +	bool want_mapped = !(freetype_flags(freetype) & FREETYPE_UNMAPPED);
> +	enum rmqueue_mode rmqm = RMQUEUE_NORMAL;
> +	unsigned long irq_flags;
> +	int nr_pageblocks, nr_freed;
> +	struct page *page;
> +	int alloc_order;
> +	int err;
> +
> +	if (freetype_idx(ft_other) < 0)
> +		return NULL;
> +
> +	/*
> +	 * Might need a TLB shootdown. Even if IRQs are on this isn't
> +	 * safe if the caller holds a lock (in case the other CPUs need that
> +	 * lock to handle the shootdown IPI).
> +	 */
> +	if (alloc_flags & ALLOC_NOBLOCK)
> +		return NULL;
> +
> +	if (!can_set_direct_map() || alloc_flags & ALLOC_NOLOCK)
> +		return NULL;
> +
> +	lockdep_assert(!irqs_disabled() || unlikely(early_boot_irqs_disabled));
> +
> +	/*
> +	 * Need to [un]map a whole pageblock (otherwise it might require
> +	 * allocating pagetables). First allocate it.
> +	 */
> +	alloc_order = max(request_order, pageblock_order);
> +	nr_pageblocks = 1 << (alloc_order - pageblock_order);
> +	spin_lock_irqsave(&zone->lock, irq_flags);
> +	/* First try a block that already has the right migratetype. */
> +	page = __rmqueue(zone, alloc_order, ft_other, alloc_flags, &rmqm);
> +	if (!page) {
> +		/* Fallback to changing a block's migratetype. */
> +		rmqm = RMQUEUE_CLAIM;
> +		page = __rmqueue(zone, alloc_order, ft_other, alloc_flags, &rmqm);
> +	}
> +	spin_unlock_irqrestore(&zone->lock, irq_flags);
> +	if (!page)
> +		return NULL;

IIUC we only try to change an entire pageblock here, but what if we
can't? If memory is fragmented enough that many pageblocks have few
unmapped pages in them, how do we serve a mapped allocation (e.g. a slab
allocation)?

We'll go into reclaim/compaction, but there's a chance we'll end up with
unexpected allocation failures or OOM kills even though we have free
memory, because unmapped memory is not movable or reclaimable (as of
now, at least).

The same could happen if many pageblocks have few mapped but unmovable
pages in them, and we make an unmapped allocation.

I wonder if we still need a fallback case where a pageblock contains a
mix of mapped and unmapped pages. We need to carefully handle such
pageblocks:
- For unmapped allocations, we need to unmap the relevant PTEs and
  potentially do a TLB shootdown (if they were previously mapped). Maybe
  we should always flush the TLB for simplicity for now.
- For mapped allocations, we need to map the relevant PTEs. No TLB
  shootdown should be needed.

Assuming unmapped allocations are always zeroed by the users on alloc
and free, we don't need to worry about zeroing pages either way.

We may want to track the number of unmapped pages in such page blocks to
now when it's fully mapped or fully unmapped and change its type, but
maybe this can be a followup if needed.