* Re: 4.8.8 kernel trigger OOM killer repeatedly when I have lots of RAM that should be free
From: Marc MERLIN @ 2017-05-02 4:12 UTC (permalink / raw)
To: Linus Torvalds
Cc: Michal Hocko, Vlastimil Babka, linux-mm, LKML, Joonsoo Kim,
Tejun Heo, Greg Kroah-Hartman
In-Reply-To: <20161129230135.GM7179@merlins.org>
Howdy,
Well, sadly, the problem is more or less back is 4.11.0. The system doesn't really
crash but it goes into an infinite loop with
[34776.826800] BUG: workqueue lockup - pool cpus=6 node=0 flags=0x0 nice=0 stuck for 33s!
More logs: https://pastebin.com/YqE4riw0
(I upgraded from 4.8 with custom patches you gave me, and went to 4.11.0
gargamel:~# cat /proc/sys/vm/dirty_ratio
2
gargamel:~# cat /proc/sys/vm/dirty_background_ratio
1
gargamel:~# free
total used free shared buffers cached
Mem: 24392600 16362660 8029940 0 8884 13739000
-/+ buffers/cache: 2614776 21777824
Swap: 15616764 0 15616764
And yet, I was doing a btrfs check repair on a busy filesystem, within 40mn or so,
it triggered the workqueue lockup.
gargamel:~# grep CONFIG_COMPACTION /boot/config-4.11.0-amd64-preempt-sysrq-20170406
CONFIG_COMPACTION=y
kernel config file: https://pastebin.com/7Tajse6L
To be fair, I didn't try to run btrfs check on 4.8 and now I'm busy
trying to recover a filesystem that apparently got corrupted by a bad
SAS driver in 4.8 which caused a lot of I/O errors and corruption.
This is just to say that btrfs on top of dmcrypt on top of bcache may
have been enough layers to hang on btrfs check on 4.8 too, but I can't
really go back to check right now due to the driver corruption issues.
Any idea what I should do next?
Thanks,
Marc
On Tue, Nov 29, 2016 at 03:01:35PM -0800, Marc MERLIN wrote:
> On Tue, Nov 29, 2016 at 09:40:19AM -0800, Marc MERLIN wrote:
> > Thanks for the reply and suggestions.
> >
> > On Tue, Nov 29, 2016 at 09:07:03AM -0800, Linus Torvalds wrote:
> > > On Tue, Nov 29, 2016 at 8:34 AM, Marc MERLIN <marc@merlins.org> wrote:
> > > > Now, to be fair, this is not a new problem, it's just varying degrees of
> > > > bad and usually only happens when I do a lot of I/O with btrfs.
> > >
> > > One situation where I've seen something like this happen is
> > >
> > > (a) lots and lots of dirty data queued up
> > > (b) horribly slow storage
> >
> > In my case, it is a 5x 4TB HDD with
> > software raid 5 < bcache < dmcrypt < btrfs
> > bcache is currently half disabled (as in I removed the actual cache) or
> > too many bcache requests pile up, and the kernel dies when too many
> > workqueues have piled up.
> > I'm just kind of worried that since I'm going through 4 subsystems
> > before my data can hit disk, that's a lot of memory allocations and
> > places where data can accumulate and cause bottlenecks if the next
> > subsystem isn't as fast.
> >
> > But this shouldn't be "horribly slow", should it? (it does copy a few
> > terabytes per day, not fast, but not horrible, about 30MB/s or so)
> >
> > > Sadly, our defaults for "how much dirty data do we allow" are somewhat
> > > buggered. The global defaults are in "percent of memory", and are
> > > generally _much_ too high for big-memory machines:
> > >
> > > [torvalds@i7 linux]$ cat /proc/sys/vm/dirty_ratio
> > > 20
> > > [torvalds@i7 linux]$ cat /proc/sys/vm/dirty_background_ratio
> > > 10
> >
> > I can confirm I have the same.
> >
> > > says that it only starts really throttling writes when you hit 20% of
> > > all memory used. You don't say how much memory you have in that
> > > machine, but if it's the same one you talked about earlier, it was
> > > 24GB. So you can have 4GB of dirty data waiting to be flushed out.
> >
> > Correct, 24GB and 4GB.
> >
> > > And we *try* to do this per-device backing-dev congestion thing to
> > > make things work better, but it generally seems to not work very well.
> > > Possibly because of inconsistent write speeds (ie _sometimes_ the SSD
> > > does really well, and we want to open up, and then it shuts down).
> > >
> > > One thing you can try is to just make the global limits much lower. As in
> > >
> > > echo 2 > /proc/sys/vm/dirty_ratio
> > > echo 1 > /proc/sys/vm/dirty_background_ratio
> >
> > I will give that a shot, thank you.
>
> And, after 5H of copying, not a single hang, or USB disconnect, or anything.
> Obviously this seems to point to other problems in the code, and I have no
> idea which layer is a culprit here, but reducing the buffers absolutely
> helped a lot.
--
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | PGP 1024R/763BE901
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v7 0/7] Introduce ZONE_CMA
From: Joonsoo Kim @ 2017-05-02 4:01 UTC (permalink / raw)
To: Michal Hocko
Cc: Andrew Morton, Rik van Riel, Johannes Weiner, mgorman,
Laura Abbott, Minchan Kim, Marek Szyprowski, Michal Nazarewicz,
Aneesh Kumar K . V, Vlastimil Babka, Russell King, Will Deacon,
linux-mm, linux-kernel, kernel-team
In-Reply-To: <20170427150636.GM4706@dhcp22.suse.cz>
On Thu, Apr 27, 2017 at 05:06:36PM +0200, Michal Hocko wrote:
> On Tue 25-04-17 12:42:57, Joonsoo Kim wrote:
> > On Mon, Apr 24, 2017 at 03:09:36PM +0200, Michal Hocko wrote:
> > > On Mon 17-04-17 11:02:12, Joonsoo Kim wrote:
> > > > On Thu, Apr 13, 2017 at 01:56:15PM +0200, Michal Hocko wrote:
> > > > > On Wed 12-04-17 10:35:06, Joonsoo Kim wrote:
> [...]
> > > not for free. For most common configurations where we have ZONE_DMA,
> > > ZONE_DMA32, ZONE_NORMAL and ZONE_MOVABLE all the 3 bits are already
> > > consumed so a new zone will need a new one AFAICS.
> >
> > Yes, it requires one more bit for a new zone and it's handled by the patch.
>
> I am pretty sure that you are aware that consuming new page flag bits
> is usually a no-go and something we try to avoid as much as possible
> because we are in a great shortage there. So there really have to be a
> _strong_ reason if we go that way. My current understanding that the
> whole zone concept is more about a more convenient implementation rather
> than a fundamental change which will solve unsolvable problems with the
> current approach. More on that below.
If there is a consensus that adding a new zone and one more bit in
page flags bits seems to be unreasonable, I try to find a way to use
ZONE_MOVABLE. As mentioned before, that's not fundamental issue to me.
However, it will have many potential problems as I mentioned so I
*really* don't prefer that way.
>
> [...]
> > MOVABLE allocation will fallback as following sequence.
> >
> > ZONE_CMA -> ZONE_MOVABLE -> ZONE_HIGHMEM -> ZONE_NORMAL -> ...
> >
> > I don't understand what you mean CMA allocation. In MM's context,
> > there is no CMA allocation. That is just MOVABLE allocation.
> >
> > For device's context, there is CMA allocation. It is range specific
> > allocation so it should be succeed for requested range. No fallback is
> > allowed in this case.
>
> OK. that answers my question. I guess... My main confusion comes from
> __alloc_gigantic_page which shares alloc_contig_range with the cma
> allocation. But from what you wrote above and my quick glance over the
> code __alloc_gigantic_page simply changes the migrate type of the pfn
> range and it doesn't move it to the zone CMA. Right?
Yes, it doesn't move it to the zone CMA.
>
> [...]
> > > > At a glance, special migratetype sound natural. I also did. However,
> > > > it's not natural in implementation POV. Zone consists of the same type
> > > > of memory (by definition ?) and MM subsystem is implemented with that
> > > > assumption. If difference type of memory shares the same zone, it easily
> > > > causes the problem and CMA problems are the such case.
> > >
> > > But this is not any different from the highmem vs. lowmem problems we
> > > already have, no? I have looked at your example in the cover where you
> > > mention utilization and the reclaim problems. With the node reclaim we
> > > will have pages from all zones on the same LRU(s). isolate_lru_pages
> > > will skip those from ZONE_CMA because their zone_idx is higher than
> > > gfp_idx(GFP_KERNEL). The same could be achieved by an explicit check for
> > > the pageblock migrate type. So the zone doesn't really help much. Or is
> > > there some aspect that I am missing?
> >
> > Your understanding is correct. It can archieved by an explict check
> > for migratetype. And, this is the main reason that we should avoid
> > such approach.
> >
> > With ZONE approach, all these things are done naturally. We don't need
> > any explicit check to anywhere. We already have a code to skip to
> > reclaim such pages by checking zone_idx.
>
> Yes, and as we have to filter pages anyway doing so for cma blocks
> doesn't sound overly burdensome from the maintenance point of view.
>
> > However, with MIGRATETYPE approach, all these things *cannot* be done
> > naturally. We need extra checks to all the places (allocator fast
> > path, reclaim path, compaction, etc...). It is really error-prone and
> > it already causes many problems due to this aspect. For the
> > performance wise, this approach is also bad since it requires to check
> > migratetype for each pages.
> >
> > Moreover, even if we adds extra checks, things cannot be easily
> > perfect.
>
> I see this point and I agree that using a specific zone might be a
> _nicer_ solution in the end but you have to consider another aspects as
> well. The main one I am worried about is a long term maintainability.
> We are really out of page flags and consuming one for a rather specific
> usecase is not good. Look at ZONE_DMA. I am pretty sure that almost
> no sane HW needs 16MB zone anymore, yet we have hard time to get rid
> of it and so we have that memory laying around unused all the time
> and blocking one page flag bit. CMA falls into a similar category
> AFAIU. I wouldn't be all that surprised if a future HW will not need CMA
> allocations in few years, yet we will have to fight to get rid of it
> like we do with ZONE_DMA. And not only that. We will also have to fight
> finding page flags for other more general usecases in the meantime.
This maintenance problem is inherent. This problem exists even if we
uses MIGRATETYPE approach. We cannot remove many hooks for CMA if a
future HW will not need CMA allocation in few years. The only
difference is that one takes single zone bit only for CMA user and the
other approach takes many hooks that we need to take care about it all
the time.
>
> > See 3) Atomic allocation failure problem. It's inherent
> > problem if we have different types of memory in a single zone.
> > We possibly can make things perfect even with MIGRATETYPE approach,
> > however, it requires additional checks in hotpath than current. It's
> > expensive and undesirable. It will make future maintenance of MM code
> > much difficult.
>
> I believe that the overhead in the hot path is not such a big deal. We
> have means to make it 0 when CMA is not used by jumplabels. I assume
> that the vast majority of systems will not use CMA. Those systems which
> use CMA should be able to cope with some slight overhead IMHO.
Please don't underestimate number of CMA user. Most of android device
uses CMA. So, there would be more devices using CMA than the server
not using CMA. They also have a right to experience the best performance.
>
> I agree that the code maintenance cost is not free. And that is a valid
> concern. CMA maintenance will not be for free in either case, though (if
> for nothing else the page flags space mentioned above). Let's see what
> what this means for mm/page_alloc.c
>
> mm/page_alloc.c | 220 ++++++++++++++++++++++++++++----------------------------
> 1 file changed, 109 insertions(+), 111 deletions(-)
>
> Not very convincing at first glance but this can be quite misleading as
> you have already mentioned because you have moved a lot of code to to
> init path. So let's just focus on the allocator hot paths
>
> @@ -800,7 +805,7 @@ static inline void __free_one_page(struct page *page,
>
> VM_BUG_ON(migratetype == -1);
> if (likely(!is_migrate_isolate(migratetype)))
> - __mod_zone_freepage_state(zone, 1 << order, migratetype);
> + __mod_zone_page_state(zone, NR_FREE_PAGES, 1 << order);
>
> VM_BUG_ON_PAGE(pfn & ((1 << order) - 1), page);
> VM_BUG_ON_PAGE(bad_range(zone, page), page);
> @@ -1804,25 +1831,11 @@ static int fallbacks[MIGRATE_TYPES][4] = {
> [MIGRATE_UNMOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_MOVABLE, MIGRATE_TYPES },
> [MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE, MIGRATE_TYPES },
> [MIGRATE_MOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_UNMOVABLE, MIGRATE_TYPES },
> -#ifdef CONFIG_CMA
> - [MIGRATE_CMA] = { MIGRATE_TYPES }, /* Never used */
> -#endif
> #ifdef CONFIG_MEMORY_ISOLATION
> [MIGRATE_ISOLATE] = { MIGRATE_TYPES }, /* Never used */
> #endif
> };
>
> -#ifdef CONFIG_CMA
> -static struct page *__rmqueue_cma_fallback(struct zone *zone,
> - unsigned int order)
> -{
> - return __rmqueue_smallest(zone, order, MIGRATE_CMA);
> -}
> -#else
> -static inline struct page *__rmqueue_cma_fallback(struct zone *zone,
> - unsigned int order) { return NULL; }
> -#endif
> -
> /*
> * Move the free pages in a range to the free lists of the requested type.
> * Note that start_page and end_pages are not aligned on a pageblock
> @@ -2090,8 +2103,7 @@ static void reserve_highatomic_pageblock(struct page *page, struct zone *zone,
>
> /* Yoink! */
> mt = get_pageblock_migratetype(page);
> - if (!is_migrate_highatomic(mt) && !is_migrate_isolate(mt)
> - && !is_migrate_cma(mt)) {
> + if (!is_migrate_highatomic(mt) && !is_migrate_isolate(mt)) {
> zone->nr_reserved_highatomic += pageblock_nr_pages;
> set_pageblock_migratetype(page, MIGRATE_HIGHATOMIC);
> move_freepages_block(zone, page, MIGRATE_HIGHATOMIC, NULL);
> @@ -2235,13 +2247,8 @@ static struct page *__rmqueue(struct zone *zone, unsigned int order,
>
> retry:
> page = __rmqueue_smallest(zone, order, migratetype);
> - if (unlikely(!page)) {
> - if (migratetype == MIGRATE_MOVABLE)
> - page = __rmqueue_cma_fallback(zone, order);
> -
> - if (!page && __rmqueue_fallback(zone, order, migratetype))
> - goto retry;
> - }
> + if (unlikely(!page) && __rmqueue_fallback(zone, order, migratetype))
> + goto retry;
>
> trace_mm_page_alloc_zone_locked(page, order, migratetype);
> return page;
> @@ -2283,9 +2290,6 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order,
> list_add_tail(&page->lru, list);
> list = &page->lru;
> alloced++;
> - if (is_migrate_cma(get_pcppage_migratetype(page)))
> - __mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
> - -(1 << order));
> }
>
> /*
> @@ -2636,10 +2640,10 @@ int __isolate_free_page(struct page *page, unsigned int order)
> * exists.
> */
> watermark = min_wmark_pages(zone) + (1UL << order);
> - if (!zone_watermark_ok(zone, 0, watermark, 0, ALLOC_CMA))
> + if (!zone_watermark_ok(zone, 0, watermark, 0, 0))
> return 0;
>
> - __mod_zone_freepage_state(zone, -(1UL << order), mt);
> + __mod_zone_page_state(zone, NR_FREE_PAGES, -(1UL << order));
> }
>
> /* Remove page from free list */
> @@ -2655,8 +2659,8 @@ int __isolate_free_page(struct page *page, unsigned int order)
> struct page *endpage = page + (1 << order) - 1;
> for (; page < endpage; page += pageblock_nr_pages) {
> int mt = get_pageblock_migratetype(page);
> - if (!is_migrate_isolate(mt) && !is_migrate_cma(mt)
> - && !is_migrate_highatomic(mt))
> + if (!is_migrate_isolate(mt) &&
> + !is_migrate_highatomic(mt))
> set_pageblock_migratetype(page,
> MIGRATE_MOVABLE);
> }
> @@ -2783,8 +2787,7 @@ struct page *rmqueue(struct zone *preferred_zone,
> spin_unlock(&zone->lock);
> if (!page)
> goto failed;
> - __mod_zone_freepage_state(zone, -(1 << order),
> - get_pcppage_migratetype(page));
> + __mod_zone_page_state(zone, NR_FREE_PAGES, -(1 << order));
>
> __count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order);
> zone_statistics(preferred_zone, zone);
> @@ -2907,12 +2910,6 @@ bool __zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
> else
> min -= min / 4;
>
> -#ifdef CONFIG_CMA
> - /* If allocation can't use CMA areas don't use free CMA pages */
> - if (!(alloc_flags & ALLOC_CMA))
> - free_pages -= zone_page_state(z, NR_FREE_CMA_PAGES);
> -#endif
> -
> /*
> * Check watermarks for an order-0 allocation request. If these
> * are not met, then a high-order request also cannot go ahead
> @@ -2940,13 +2937,6 @@ bool __zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
> if (!list_empty(&area->free_list[mt]))
> return true;
> }
> -
> -#ifdef CONFIG_CMA
> - if ((alloc_flags & ALLOC_CMA) &&
> - !list_empty(&area->free_list[MIGRATE_CMA])) {
> - return true;
> - }
> -#endif
> }
> return false;
> }
> @@ -2962,13 +2952,6 @@ static inline bool zone_watermark_fast(struct zone *z, unsigned int order,
> unsigned long mark, int classzone_idx, unsigned int alloc_flags)
> {
> long free_pages = zone_page_state(z, NR_FREE_PAGES);
> - long cma_pages = 0;
> -
> -#ifdef CONFIG_CMA
> - /* If allocation can't use CMA areas don't use free CMA pages */
> - if (!(alloc_flags & ALLOC_CMA))
> - cma_pages = zone_page_state(z, NR_FREE_CMA_PAGES);
> -#endif
>
> /*
> * Fast check for order-0 only. If this fails then the reserves
> @@ -2977,7 +2960,7 @@ static inline bool zone_watermark_fast(struct zone *z, unsigned int order,
> * the caller is !atomic then it'll uselessly search the free
> * list. That corner case is then slower but it is harmless.
> */
> - if (!order && (free_pages - cma_pages) > mark + z->lowmem_reserve[classzone_idx])
> + if (!order && free_pages > mark + z->lowmem_reserve[classzone_idx])
> return true;
>
> return __zone_watermark_ok(z, order, mark, classzone_idx, alloc_flags,
> @@ -3547,10 +3530,6 @@ gfp_to_alloc_flags(gfp_t gfp_mask)
> } else if (unlikely(rt_task(current)) && !in_interrupt())
> alloc_flags |= ALLOC_HARDER;
>
> -#ifdef CONFIG_CMA
> - if (gfpflags_to_migratetype(gfp_mask) == MIGRATE_MOVABLE)
> - alloc_flags |= ALLOC_CMA;
> -#endif
> return alloc_flags;
> }
>
> @@ -3972,9 +3951,6 @@ static inline bool prepare_alloc_pages(gfp_t gfp_mask, unsigned int order,
> if (should_fail_alloc_page(gfp_mask, order))
> return false;
>
> - if (IS_ENABLED(CONFIG_CMA) && ac->migratetype == MIGRATE_MOVABLE)
> - *alloc_flags |= ALLOC_CMA;
> -
> return true;
> }
>
> This looks like a nice clean up. Those ifdefs are ugly as hell. One
> could argue that some of that could be cleaned up by simply adding some
> helpers (with a jump label to reduce the overhead), though. But is this
> really strong enough reason to bring the whole zone in? I am not really
> convinced to be honest.
Please don't underestimate the benefit of this patchset.
I have said that we need *more* hooks to fix all the problems.
And, please think that this code removal is not only code removal but
also concept removal. With this removing, we don't need to consider
ALLOC_CMA for alloc_flags when calling zone_watermark_ok(). There are
many bugs on it and it still remains. We don't need to consider
pageblock migratetype when handling pageblock migratetype. We don't
need to take a great care about calculating the number of CMA
freepages.
>
> [...]
>
> > > Please do _not_ take this as a NAK from me. At least not at this time. I
> > > am still trying to understand all the consequences but my intuition
> > > tells me that building on top of highmem like approach will turn out to
> > > be problematic in future (as we have already seen with the highmem and
> > > movable zones) so this needs a very prudent consideration.
> >
> > I can understand that you are prudent to this issue. However, it takes more
> > than two years and many people already expressed that ZONE approach is the
> > way to go.
>
> I can see a single Acked-by and one Reviewed-by. It would be much more
> convincing to see much larger support. Do not take me wrong I am not
> trying to undermine the feedback so far but we should be clear about one
> thing. CMA is mostly motivated by the industry which tries to overcome
> HW limitations which can change in future very easily. I would rather
> see good enough solution for something like that than a nicer solution
> which is pushing additional burden on more general usecases.
First of all, current MIGRATETYPE approach isn't good enough to me.
They caused too many problems and there are many remanining problems.
It will causes maintenance issue for a long time.
And, although good enough solution can be better than nicer solution
in some cases, it looks like current situation isn't that case.
There is a single reason, saving page flag bit, to support good enough
solution.
I'd like to ask reversly. Is this a enough reason to make CMA user to
suffer from bugs? Is it a enough reason to maintain such dirty and
hard maintainable code, and to hurt other MM devepoler's development
performance? We need a great care on handling pageblock migratetype,
accounting freepage, reclaiming the allocated page. It would affect
many developers.
>
> That being said, I would like to see a much larger consensus in the MM
> community before a new zone is merged. I am staying very skeptical this
> is the right direction though.
As you know, CMA is actively used in industry and they don't review
the patch in public mailing list. Moreover, there is no interest on
CMA by core MM developers. So, I think that 1 acked-by and 1
reviewed-by is a great agreement. I and they are most of developers
who have interest on CMA in MM community.
Anyway, AFAIK, following people are in favor of this patchset.
If I am wrong, please let me know.
Minchan Kim <minchan@kernel.org>
Laura Abbot <lauraa@codeaurora.org>
Bob Liu <liubo95@huawei.com>
As you know (?), Minchan and Laura are also the core developers to CMA
topic.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RESENT PATCH] x86/mem: fix the offset overflow when read/write mem
From: zhong jiang @ 2017-05-02 1:47 UTC (permalink / raw)
To: akpm
Cc: arnd, hannes, kirill, rientjes, mgorman, hughd, riel, linux-mm,
linux-kernel
In-Reply-To: <1493293775-57176-1-git-send-email-zhongjiang@huawei.com>
ping ....
anyone has any objections.
On 2017/4/27 19:49, zhongjiang wrote:
> From: zhong jiang <zhongjiang@huawei.com>
>
> Recently, I found the following issue, it will result in the panic.
>
> [ 168.739152] mmap1: Corrupted page table at address 7f3e6275a002
> [ 168.745039] PGD 61f4a1067
> [ 168.745040] PUD 61ab19067
> [ 168.747730] PMD 61fb8b067
> [ 168.750418] PTE 8000100000000225
> [ 168.753109]
> [ 168.757795] Bad pagetable: 000d [#1] SMP
> [ 168.761696] Modules linked in: intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc aesni_intel crypto_simd iTCO_wdt glue_helper cryptd sg iTCO_vendor_support i7core_edac edac_core shpchp lpc_ich i2c_i801 pcspkr mfd_core acpi_cpufreq ip_tables xfs libcrc32c sd_mod igb ata_generic ptp pata_acpi pps_core mptsas ata_piix scsi_transport_sas i2c_algo_bit libata mptscsih i2c_core serio_raw crc32c_intel bnx2 mptbase dca dm_mirror dm_region_hash dm_log dm_mod
> [ 168.805983] CPU: 15 PID: 10369 Comm: mmap1 Not tainted 4.11.0-rc2-327.28.3.53.x86_64+ #345
> [ 168.814202] Hardware name: Huawei Technologies Co., Ltd. Tecal RH2285 /BC11BTSA , BIOS CTSAV036 04/27/2011
> [ 168.825704] task: ffff8806207d5200 task.stack: ffffc9000c340000
> [ 168.831592] RIP: 0033:0x7f3e622c5360
> [ 168.835147] RSP: 002b:00007ffe2bb7a098 EFLAGS: 00010203
> [ 168.840344] RAX: 00007ffe2bb7a0c0 RBX: 0000000000000000 RCX: 00007f3e6275a000
> [ 168.847439] RDX: 00007f3e622c5360 RSI: 00007f3e6275a000 RDI: 00007ffe2bb7a0c0
> [ 168.854535] RBP: 00007ffe2bb7a4e0 R08: 00007f3e621c3d58 R09: 000000000000002d
> [ 168.861632] R10: 00007ffe2bb79e20 R11: 00007f3e622fbcb0 R12: 00000000004005d0
> [ 168.868728] R13: 00007ffe2bb7a5c0 R14: 0000000000000000 R15: 0000000000000000
> [ 168.875825] FS: 00007f3e62752740(0000) GS:ffff880627bc0000(0000) knlGS:0000000000000000
> [ 168.883870] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 168.889583] CR2: 00007f3e6275a002 CR3: 0000000622845000 CR4: 00000000000006e0
> [ 168.896680] RIP: 0x7f3e622c5360 RSP: 00007ffe2bb7a098
> [ 168.901713] ---[ end trace ef98fa9f2a01cbc6 ]---
> [ 168.90630 arch/x86/kernel/smp.c:127 native_smp_send_reschedule+0x3f/0x50
> [ 168.935410] Modules linked in: intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc aesni_intel crypto_simd iTCO_wdt glue_helper cryptd sg iTCO_vendor_support i7core_edac edac_core shpchp lpc_ich i2c_i801 pcspkr mfd_core acpi_cpufreq ip_tables xfs libcrc32c sd_mod igb ata_generic ptp pata_acpi pps_core mptsas ata_piix scsi_transport_sas i2c_algo_bit libata mptscsih i2c_core serio_raw crc32c_intel bnx2 mptbase dca dm_mirror dm_region_hash dm_log dm_mod
> [ 168.979686] CPU: 15 PID: 10369 Comm: mmap1 Tainted: G D 4.11.0-rc2-327.28.3.53.x86_64+ #345
> [ 168.989114] Hardware name: Huawei Technologies Co., Ltd. Tecal RH2285 /BC11BTSA , BIOS CTSAV036 04/27/2011
> [ 169.000616] Call Trace:
> [ 169.003049] <IRQ>
> [ 169.005050] dump_stack+0x63/0x84
> [ 169.008348] __warn+0xd1/0xf0
> [ 169.011297] warn_slowpath_null+0x1d/0x20
> [ 169.015282] native_smp_send_reschedule+0x3f/0x50
> [ 169.019961] resched_curr+0xa1/0xc0
> [ 169.023428] check_preempt_curr+0x70/0x90
> [ 169.027415] ttwu_do_wakeup+0x1e/0x160
> [ 169.031142] ttwu_do_activate+0x77/0x80
> [ 169.034956] try_to_wake_up+0x1c3/0x430
> [ 169.038771] default_wake_function+0x12/0x20
> [ 169.043019] __wake_up_common+0x55/0x90
> [ 169.046833] __wake_up_locked+0x13/0x20
> [ 169.050649] ep_poll_callback+0xbb/0x240
> [ 169.054550] __wake_up_common+0x55/0x90
> [ 169.058363] __wake_up+0x39/0x50
> [ 169.061574] wake_up_klogd_work_func+0x40/0x60
> [ 169.065994] irq_work_run_list+0x4d/0x70
> [ 169.069895] irq_work_tick+0x40/0x50
> [ 169.073452] update_process_times+0x42/0x60
> [ 169.077612] tick_periodic+0x2b/0x80
> [ 169.081166] tick_handle_periodic+0x25/0x70
> [ 169.085326] local_apic_timer_interrupt+0x35/0x60
> [ 169.090004] smp_apic_timer_interrupt+0x38/0x50
> [ 169.094507] apic_timer_interrupt+0x93/0xa0
> [ 169.098667] RIP: 0010:panic+0x1f5/0x239
> [ 169.102480] RSP: 0000:ffffc9000c343dd8 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff10
> [ 169.110010] RAX: 0000000000000034 RBX: 0000000000000000 RCX: 0000000000000006
> [ 169.117106] RDX: 0000000000000000 RSI: 0000000000000086 RDI: ffff880627bcdfe0
> [ 169.124201] RBP: ffffc9000c343e48 R08: 00000000fffffffe R09: 0000000000000395
> [ 169.131298] R10: 0000000000000005 R11: 0000000000000394 R12: ffffffff81a0c475
> [ 169.138395] R13: 0000000000000000 R14: 0000000000000000 R15: 000000000000000d
> [ 169.145491] </IRQ>
> [ 169.147578] ? panic+0x1f1/0x239
> [ 169.150789] oops_end+0xb8/0xd0
> [ 169.153910] pgtable_bad+0x8a/0x95
> [ 169.157294] __do_page_fault+0x3aa/0x4a0
> [ 169.161194] do_page_fault+0x30/0x80
> [ 169.164750] ? do_syscall_64+0x175/0x180
> [ 169.168649] page_fault+0x28/0x30
>
> the following case can reproduce the issue.
>
> int mem_fd = 0;
> char rw_buf[1024];
> unsigned char * map_base_s;
> unsigned long show_addr = 0x100000000000;
> unsigned long show_len = 0x10;
>
> if(argc !=2 )
> {
> printf( "%s show_addr\n", argv[0] );
> return 0;
> }
> else
> {
> char *stop;
> show_addr = strtoul( argv[1], &stop, 0 );
> printf("show_addr= 0x%lu\n", show_addr );
> }
>
> mem_fd = open(DEV_NAME, O_RDONLY);
> if (mem_fd == -1)
> {
> printf("open %s failed.", DEV_NAME);
> return 0;
> }
>
> map_base_s = mmap(NULL, show_len, PROT_READ, MAP_SHARED, mem_fd, show_addr);
> if ((long)map_base_s == -1)
> {
> printf("input address map to user space fail!\n");
> return 0;
> }
> else
> {
> printf("mmap successfull!\n");
> }
>
> memcpy( rw_buf, map_base_s, show_len );
>
> The pgoff is enough large, it exceed the size of the real memory.
> and the mmap can return the success.
>
> I fix it by checking the conditions. it can make it suitable for
> the mapped and use.
>
> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
> ---
> drivers/char/mem.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/char/mem.c b/drivers/char/mem.c
> index 7e4a9d1..3a765e02 100644
> --- a/drivers/char/mem.c
> +++ b/drivers/char/mem.c
> @@ -55,7 +55,7 @@ static inline int valid_phys_addr_range(phys_addr_t addr, size_t count)
>
> static inline int valid_mmap_phys_addr_range(unsigned long pfn, size_t size)
> {
> - return 1;
> + return (pfn << PAGE_SHIFT) + size <= __pa(high_memory);
> }
> #endif
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: Balbir Singh @ 2017-05-02 1:29 UTC (permalink / raw)
To: John Hubbard, linux-mm, akpm
Cc: khandual, benh, aneesh.kumar, paulmck, srikar, haren, jglisse,
mgorman, mhocko, arbab, vbabka, cl
In-Reply-To: <91272c14-81df-9529-f0ae-6abb17a694ea@nvidia.com>
On Mon, 2017-05-01 at 13:41 -0700, John Hubbard wrote:
> On 04/19/2017 12:52 AM, Balbir Singh wrote:
> > This is a request for comments on the discussed approaches
> > for coherent memory at mm-summit (some of the details are at
> > https://lwn.net/Articles/717601/). The latest posted patch
> > series is at https://lwn.net/Articles/713035/. I am reposting
> > this as RFC, Michal Hocko suggested using HMM for CDM, but
> > we believe there are stronger reasons to use the NUMA approach.
> > The earlier patches for Coherent Device memory were implemented
> > and designed by Anshuman Khandual.
> >
>
> Hi Balbir,
>
> Although I think everyone agrees that in the [very] long term, these
> hardware-coherent nodes probably want to be NUMA nodes, in order to decide what to
> code up over the next few years, we need to get a clear idea of what has to be done
> for each possible approach.
>
> Here, the CDM discussion is falling just a bit short, because it does not yet
> include the whole story of what we would need to do. Earlier threads pointed this
> out: the idea started as a large patchset RFC, but then, "for ease of review", it
> got turned into a smaller RFC, which loses too much context.
Hi, John
I thought I explained the context, but I'll try again. I see the whole solution
as a composite of the following primitives:
1. Enable hotplug of CDM nodes
2. Isolation of CDM memory
3. Migration to/from CDM memory
4. Performance enhancements for migration
The RFC here is for (2) above. (3) is handled by HMM and (4) is being discussed
in the community. I think the larger goals are same as HMM, except that we
don't need unaddressable memory, since the memory is cache coherent.
>
> So, I'd suggest putting together something more complete, so that it can be fairly
> compared against the HMM-for-hardware-coherent-nodes approach.
>
Since I intend to reuse bits of HMM, I am not sure if I want to repost those
patches as a part of my RFC. I hope my answers make sense, the goal is to
reuse as much of what is available. From a user perspective
1. We see no new interface being added in either case, the programming model
would differ though
2. We expect the programming model to be abstracted behind a user space
framework, potentially like CUDA or CXL
>
> > Jerome posted HMM-CDM at https://lwn.net/Articles/713035/.
> > The patches do a great deal to enable CDM with HMM, but we
> > still believe that HMM with CDM is not a natural way to
> > represent coherent device memory and the mm will need
> > to be audited and enhanced for it to even work.
>
> That is also true for the CDM approach. Specifically, in order for this to be of any
> use to device drivers, we'll need the following:
>
Since Reza answered these questions, I'll skip them in this email
Thanks for the review!
Balbir Singh
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: Reza Arbab @ 2017-05-02 0:04 UTC (permalink / raw)
To: John Hubbard
Cc: Balbir Singh, linux-mm, akpm, khandual, benh, aneesh.kumar,
paulmck, srikar, haren, jglisse, mgorman, mhocko, vbabka, cl
In-Reply-To: <d7e4b032-0c73-92fa-9c70-fbda98df849c@nvidia.com>
On Mon, May 01, 2017 at 04:58:14PM -0700, John Hubbard wrote:
>On 05/01/2017 04:51 PM, Reza Arbab wrote:
>>On Mon, May 01, 2017 at 02:56:34PM -0700, John Hubbard wrote:
>>>On 05/01/2017 02:04 PM, Reza Arbab wrote:
>>>>On Mon, May 01, 2017 at 01:41:55PM -0700, John Hubbard wrote:
>>>>>1. A way to move pages between NUMA nodes, both virtual
>>>>>address and physical address-based, from kernel mode.
>>>>
>>>>Jerome's migrate_vma() and migrate_dma() should have this
>>>>covered, including DMA-accelerated copy.
>>>
>>>Yes, that's good. I wasn't sure from this discussion here if
>>>either or both of those would be used, but now I see.
>>>
>>>Are those APIs ready for moving pages between NUMA nodes? As there
>>>is no NUMA node id in the API, are we relying on the pages'
>>>membership (using each page and updating which node it is on)?
>>
>>Yes. Those APIs work by callback. The alloc_and_copy() function you
>>provide will be called at the appropriate point in the migration.
>>Yours would allocate from a specific destination node, and copy
>>using DMA.
>>
>
>hmmm, that reminds me: the whole story of "which device is this, and
>which NUMA node does it correlate to?" will have to be wired up. That
>is *probably* all in the device driver, but since I haven't worked
>through it, I'd be inclined to list it as an item on the checklist,
>just in case it requires some little hook in the upstream kernel.
I've worked this out. It can be contained to the driver itself.
--
Reza Arbab
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: John Hubbard @ 2017-05-01 23:58 UTC (permalink / raw)
To: Reza Arbab
Cc: Balbir Singh, linux-mm, akpm, khandual, benh, aneesh.kumar,
paulmck, srikar, haren, jglisse, mgorman, mhocko, vbabka, cl
In-Reply-To: <20170501235123.2k372i75vxlw5n75@arbab-vm>
On 05/01/2017 04:51 PM, Reza Arbab wrote:
> On Mon, May 01, 2017 at 02:56:34PM -0700, John Hubbard wrote:
>> On 05/01/2017 02:04 PM, Reza Arbab wrote:
>>> On Mon, May 01, 2017 at 01:41:55PM -0700, John Hubbard wrote:
>>>> 1. A way to move pages between NUMA nodes, both virtual address and physical
>>>> address-based, from kernel mode.
>>>
>>> Jérôme's migrate_vma() and migrate_dma() should have this covered, including
>>> DMA-accelerated copy.
>>
>> Yes, that's good. I wasn't sure from this discussion here if either or both of
>> those would be used, but now I see.
>>
>> Are those APIs ready for moving pages between NUMA nodes? As there is no NUMA node
>> id in the API, are we relying on the pages' membership (using each page and
>> updating which node it is on)?
>
> Yes. Those APIs work by callback. The alloc_and_copy() function you provide will be
> called at the appropriate point in the migration. Yours would allocate from a
> specific destination node, and copy using DMA.
>
hmmm, that reminds me: the whole story of "which device is this, and which NUMA node
does it correlate to?" will have to be wired up. That is *probably* all in the
device driver, but since I haven't worked through it, I'd be inclined to list it as
an item on the checklist, just in case it requires some little hook in the upstream
kernel.
thanks,
john h
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH -mm -v10 1/3] mm, THP, swap: Delay splitting THP during swap out
From: Minchan Kim @ 2017-05-01 23:53 UTC (permalink / raw)
To: Johannes Weiner
Cc: Huang, Ying, Andrew Morton, linux-mm, linux-kernel,
Andrea Arcangeli, Ebru Akagunduz, Michal Hocko, Tejun Heo,
Hugh Dickins, Shaohua Li, Rik van Riel, cgroups
In-Reply-To: <20170501104430.GA16306@cmpxchg.org>
Hi Johannes,
The patch I sent has two clean-up.
First part was as follows:
^ permalink raw reply
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: Reza Arbab @ 2017-05-01 23:51 UTC (permalink / raw)
To: John Hubbard
Cc: Balbir Singh, linux-mm, akpm, khandual, benh, aneesh.kumar,
paulmck, srikar, haren, jglisse, mgorman, mhocko, vbabka, cl
In-Reply-To: <ce589129-d86c-ba43-7e04-55acf08f7f29@nvidia.com>
On Mon, May 01, 2017 at 02:56:34PM -0700, John Hubbard wrote:
>On 05/01/2017 02:04 PM, Reza Arbab wrote:
>>On Mon, May 01, 2017 at 01:41:55PM -0700, John Hubbard wrote:
>>>1. A way to move pages between NUMA nodes, both virtual address
>>>and physical address-based, from kernel mode.
>>
>>Jerome's migrate_vma() and migrate_dma() should have this covered,
>>including DMA-accelerated copy.
>
>Yes, that's good. I wasn't sure from this discussion here if either or
>both of those would be used, but now I see.
>
>Are those APIs ready for moving pages between NUMA nodes? As there is
>no NUMA node id in the API, are we relying on the pages' membership
>(using each page and updating which node it is on)?
Yes. Those APIs work by callback. The alloc_and_copy() function you
provide will be called at the appropriate point in the migration. Yours
would allocate from a specific destination node, and copy using DMA.
>>>5. Something to handle the story of bringing NUMA nodes online and
>>>putting them back offline, given that they require a device driver
>>>that may not yet have been loaded. There are a few minor missing bits
>>>there.
>>
>>This has been prototyped with the driver doing memory
>>hotplug/hotremove. Could you elaborate a little on what you feel is
>>missing?
>>
>
>We just worked through how to deal with this in our driver, and I
>remember feeling worried about the way NUMA nodes can only be put
>online via a user space action (through sysfs). It seemed like you'd
>want to do that from kernel as well, when a device driver gets loaded.
That's true. I don't think we have a way to online/offline from a
driver. To online, the alternatives are memhp_auto_online (incapable of
doing online_movable), or udev rules (not ideal in this driver
controlled memory use case). To offline, nothing that I know of.
>I was also uneasy about user space trying to bring a node online before
>the associated device driver was loaded, and I think it would be nice
>to be sure that that whole story is looked at.
>
>The theme here is that driver load/unload is, today, independent from
>the NUMA node online/offline, and that's a problem. Not a huge one,
>though, just worth enumerating here.
--
Reza Arbab
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 2/2] dax: fix data corruption due to stale mmap reads
From: Dan Williams @ 2017-05-01 22:59 UTC (permalink / raw)
To: Jan Kara
Cc: Ross Zwisler, Andrew Morton, linux-kernel@vger.kernel.org,
Alexander Viro, Alexey Kuznetsov, Andrey Ryabinin, Anna Schumaker,
Christoph Hellwig, Darrick J. Wong, Eric Van Hensbergen,
Jens Axboe, Johannes Weiner, Konrad Rzeszutek Wilk,
Latchesar Ionkov, linux-cifs, linux-fsdevel, Linux MM, linux-nfs,
linux-nvdimm@lists.01.org, Matthew Wilcox, Ron Minnich,
samba-technical, Steve French, Trond Myklebust, v9fs-developer
In-Reply-To: <20170427072659.GA29789@quack2.suse.cz>
On Thu, Apr 27, 2017 at 12:26 AM, Jan Kara <jack@suse.cz> wrote:
> On Wed 26-04-17 16:52:36, Ross Zwisler wrote:
>> On Wed, Apr 26, 2017 at 10:52:35AM +0200, Jan Kara wrote:
>> > On Tue 25-04-17 16:59:36, Ross Zwisler wrote:
>> > > On Tue, Apr 25, 2017 at 01:10:43PM +0200, Jan Kara wrote:
>> > > <>
>> > > > Hum, but now thinking more about it I have hard time figuring out why write
>> > > > vs fault cannot actually still race:
>> > > >
>> > > > CPU1 - write(2) CPU2 - read fault
>> > > >
>> > > > dax_iomap_pte_fault()
>> > > > ->iomap_begin() - sees hole
>> > > > dax_iomap_rw()
>> > > > iomap_apply()
>> > > > ->iomap_begin - allocates blocks
>> > > > dax_iomap_actor()
>> > > > invalidate_inode_pages2_range()
>> > > > - there's nothing to invalidate
>> > > > grab_mapping_entry()
>> > > > - we add zero page in the radix
>> > > > tree & map it to page tables
>> > > >
>> > > > Similarly read vs write fault may end up racing in a wrong way and try to
>> > > > replace already existing exceptional entry with a hole page?
>> > >
>> > > Yep, this race seems real to me, too. This seems very much like the issues
>> > > that exist when a thread is doing direct I/O. One thread is doing I/O to an
>> > > intermediate buffer (page cache for direct I/O case, zero page for us), and
>> > > the other is going around it directly to media, and they can get out of sync.
>> > >
>> > > IIRC the direct I/O code looked something like:
>> > >
>> > > 1/ invalidate existing mappings
>> > > 2/ do direct I/O to media
>> > > 3/ invalidate mappings again, just in case. Should be cheap if there weren't
>> > > any conflicting faults. This makes sure any new allocations we made are
>> > > faulted in.
>> >
>> > Yeah, the problem is people generally expect weird behavior when they mix
>> > direct and buffered IO (or let alone mmap) however everyone expects
>> > standard read(2) and write(2) to be completely coherent with mmap(2).
>>
>> Yep, fair enough.
>>
>> > > I guess one option would be to replicate that logic in the DAX I/O path, or we
>> > > could try and enhance our locking so page faults can't race with I/O since
>> > > both can allocate blocks.
>> >
>> > In the abstract way, the problem is that we have radix tree (and page
>> > tables) cache block mapping information and the operation: "read block
>> > mapping information, store it in the radix tree" is not serialized in any
>> > way against other block allocations so the information we store can be out
>> > of date by the time we store it.
>> >
>> > One way to solve this would be to move ->iomap_begin call in the fault
>> > paths under entry lock although that would mean I have to redo how ext4
>> > handles DAX faults because with current code it would create lock inversion
>> > wrt transaction start.
>>
>> I don't think this alone is enough to save us. The I/O path doesn't currently
>> take any DAX radix tree entry locks, so our race would just become:
>>
>> CPU1 - write(2) CPU2 - read fault
>>
>> dax_iomap_pte_fault()
>> grab_mapping_entry() // newly moved
>> ->iomap_begin() - sees hole
>> dax_iomap_rw()
>> iomap_apply()
>> ->iomap_begin - allocates blocks
>> dax_iomap_actor()
>> invalidate_inode_pages2_range()
>> - there's nothing to invalidate
>> - we add zero page in the radix
>> tree & map it to page tables
>>
>> In their current form I don't think we want to take DAX radix tree entry locks
>> in the I/O path because that would effectively serialize I/O over a given
>> radix tree entry. For a 2MiB entry, for example, all I/O to that 2MiB range
>> would be serialized.
>
> Note that invalidate_inode_pages2_range() will see the entry created by
> grab_mapping_entry() on CPU2 and block waiting for its lock and this is
> exactly what stops the race. The invalidate_inode_pages2_range()
> effectively makes sure there isn't any page fault in progress for given
> range...
>
> Also note that writes to a file are serialized by i_rwsem anyway (and at
> least serialization of writes to the overlapping range is required by POSIX)
> so this doesn't add any more serialization than we already have.
>
>> > Another solution would be to grab i_mmap_sem for write when doing write
>> > fault of a page and similarly have it grabbed for writing when doing
>> > write(2). This would scale rather poorly but if we later replaced it with a
>> > range lock (Davidlohr has already posted a nice implementation of it) it
>> > won't be as bad. But I guess option 1) is better...
>>
>> The best idea I had for handling this sounds similar, which would be to
>> convert the radix tree locks to essentially be reader/writer locks. I/O and
>> faults that don't modify the block mapping could just take read-level locks,
>> and could all run concurrently. I/O or faults that modify a block mapping
>> would take a write lock, and serialize with other writers and readers.
>
> Well, this would be difficult to implement inside the radix tree (not
> enough bits in the entry) so you'd have to go for some external locking
> primitive anyway. And if you do that, read-write range lock Davidlohr has
> implemented is what you describe - well we could also have a radix tree
> with rwsems but I suspect the overhead of maintaining that would be too
> large. It would require larger rewrite than reusing entry locks as I
> suggest above though and it isn't an obvious performance win for realistic
> workloads either so I'd like to see some performance numbers before going
> that way. It likely improves a situation where processes race to fault the
> same page for which we already know the block mapping but I'm not sure if
> that translates to any measurable performance wins for workloads on DAX
> filesystem.
I'm also concerned about inventing new / fancy radix infrastructure
when we're already in the space of needing struct page for any
non-trivial usage of dax. As Kirill's transparent-huge-page page cache
implementation matures I'd be interested in looking at a transition
path away from radix locking towards something that it shared with the
common case page cache locking.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 2/2] dax: fix data corruption due to stale mmap reads
From: Ross Zwisler @ 2017-05-01 22:38 UTC (permalink / raw)
To: Jan Kara
Cc: Ross Zwisler, Andrew Morton, linux-kernel, Alexander Viro,
Alexey Kuznetsov, Andrey Ryabinin, Anna Schumaker,
Christoph Hellwig, Dan Williams, Darrick J. Wong,
Eric Van Hensbergen, Jens Axboe, Johannes Weiner,
Konrad Rzeszutek Wilk, Latchesar Ionkov, linux-cifs,
linux-fsdevel, linux-mm, linux-nfs, linux-nvdimm, Matthew Wilcox,
Ron Minnich, samba-technical, Steve French, Trond Myklebust,
v9fs-developer
In-Reply-To: <20170427072659.GA29789@quack2.suse.cz>
On Thu, Apr 27, 2017 at 09:26:59AM +0200, Jan Kara wrote:
> On Wed 26-04-17 16:52:36, Ross Zwisler wrote:
<>
> > I don't think this alone is enough to save us. The I/O path doesn't currently
> > take any DAX radix tree entry locks, so our race would just become:
> >
> > CPU1 - write(2) CPU2 - read fault
> >
> > dax_iomap_pte_fault()
> > grab_mapping_entry() // newly moved
> > ->iomap_begin() - sees hole
> > dax_iomap_rw()
> > iomap_apply()
> > ->iomap_begin - allocates blocks
> > dax_iomap_actor()
> > invalidate_inode_pages2_range()
> > - there's nothing to invalidate
> > - we add zero page in the radix
> > tree & map it to page tables
> >
> > In their current form I don't think we want to take DAX radix tree entry locks
> > in the I/O path because that would effectively serialize I/O over a given
> > radix tree entry. For a 2MiB entry, for example, all I/O to that 2MiB range
> > would be serialized.
>
> Note that invalidate_inode_pages2_range() will see the entry created by
> grab_mapping_entry() on CPU2 and block waiting for its lock and this is
> exactly what stops the race. The invalidate_inode_pages2_range()
> effectively makes sure there isn't any page fault in progress for given
> range...
Yep, this is the bit that I was missing. Thanks.
> Also note that writes to a file are serialized by i_rwsem anyway (and at
> least serialization of writes to the overlapping range is required by POSIX)
> so this doesn't add any more serialization than we already have.
>
> > > Another solution would be to grab i_mmap_sem for write when doing write
> > > fault of a page and similarly have it grabbed for writing when doing
> > > write(2). This would scale rather poorly but if we later replaced it with a
> > > range lock (Davidlohr has already posted a nice implementation of it) it
> > > won't be as bad. But I guess option 1) is better...
> >
> > The best idea I had for handling this sounds similar, which would be to
> > convert the radix tree locks to essentially be reader/writer locks. I/O and
> > faults that don't modify the block mapping could just take read-level locks,
> > and could all run concurrently. I/O or faults that modify a block mapping
> > would take a write lock, and serialize with other writers and readers.
>
> Well, this would be difficult to implement inside the radix tree (not
> enough bits in the entry) so you'd have to go for some external locking
> primitive anyway. And if you do that, read-write range lock Davidlohr has
> implemented is what you describe - well we could also have a radix tree
> with rwsems but I suspect the overhead of maintaining that would be too
> large. It would require larger rewrite than reusing entry locks as I
> suggest above though and it isn't an obvious performance win for realistic
> workloads either so I'd like to see some performance numbers before going
> that way. It likely improves a situation where processes race to fault the
> same page for which we already know the block mapping but I'm not sure if
> that translates to any measurable performance wins for workloads on DAX
> filesystem.
>
> > You could know if you needed a write lock without asking the filesystem - if
> > you're a write and the radix tree entry is empty or is for a zero page, you
> > grab the write lock.
> >
> > This dovetails nicely with the idea of having the radix tree act as a cache
> > for block mappings. You take the appropriate lock on the radix tree entry,
> > and it has the block mapping info for your I/O or fault so you don't have to
> > call into the FS. I/O would also participate so we would keep info about
> > block mappings that we gather from I/O to help shortcut our page faults.
> >
> > How does this sound vs the range lock idea? How hard do you think it would be
> > to convert our current wait queue system to reader/writer style locking?
> >
> > Also, how do you think we should deal with the current PMD corruption? Should
> > we go with the current fix (I can augment the comments as you suggested), and
> > then handle optimizations to that approach and the solution to this larger
> > race as a follow-on?
>
> So for now I'm still more inclined to just stay with the radix tree lock as
> is and just fix up the locking as I suggest and go for larger rewrite only
> if we can demonstrate further performance wins.
Sounds good.
> WRT your second patch, if we go with the locking as I suggest, it is enough
> to unmap the whole range after invalidate_inode_pages2() has cleared radix
> tree entries (*) which will be much cheaper (for large writes) than doing
> unmapping entry by entry.
I'm still not convinced that it is safe to do the unmap in a separate step. I
see your point about it being expensive to do a rmap walk to unmap each entry
in __dax_invalidate_mapping_entry(), but I think we might need to because the
unmap is part of the contract imposed by invalidate_inode_pages2_range() and
invalidate_inode_pages2(). This exists in the header comment above each:
* Any pages which are found to be mapped into pagetables are unmapped prior
* to invalidation.
If you look at the usage of invalidate_inode_pages2_range() in
generic_file_direct_write() for example (which I realize we won't call for a
DAX inode, but still), I think that it really does rely on the fact that
invalidated pages are unmapped, right? If it didn't, and hole pages were
mapped, the hole pages could remain mapped while a direct I/O write allocated
blocks and then wrote real data.
If we really want to unmap the entire range at once, maybe it would have to be
done in invalidate_inode_pages2_range(), after the loop? My hesitation about
this is that we'd be leaking yet more DAX special casing up into the
mm/truncate.c code.
Or am I missing something?
> So I'd go for that. I'll prepare a patch for the
> locking change - it will require changes to ext4 transaction handling so it
> won't be completely trivial.
>
> (*) The flow of information is: filesystem block mapping info -> radix tree
> -> page tables so if 'filesystem block mapping info' changes, we should go
> invalidate corresponding radix tree entries (new entries will already have
> uptodate info) and then invalidate corresponding page tables (again once
> radix tree has no stale entries, we are sure new page table entries will be
> uptodate).
>
> Honza
> --
> Jan Kara <jack@suse.com>
> SUSE Labs, CR
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: John Hubbard @ 2017-05-01 21:56 UTC (permalink / raw)
To: Reza Arbab
Cc: Balbir Singh, linux-mm, akpm, khandual, benh, aneesh.kumar,
paulmck, srikar, haren, jglisse, mgorman, mhocko, vbabka, cl
In-Reply-To: <20170501210415.aeuvd73auomvdmba@arbab-laptop.localdomain>
On 05/01/2017 02:04 PM, Reza Arbab wrote:
> On Mon, May 01, 2017 at 01:41:55PM -0700, John Hubbard wrote:
>> 1. A way to move pages between NUMA nodes, both virtual address and physical
>> address-based, from kernel mode.
>
> Jérôme's migrate_vma() and migrate_dma() should have this covered, including
> DMA-accelerated copy.
Yes, that's good. I wasn't sure from this discussion here if either or both of those
would be used, but now I see.
Are those APIs ready for moving pages between NUMA nodes? As there is no NUMA node
id in the API, are we relying on the pages' membership (using each page and updating
which node it is on)?
>
>> 5. Something to handle the story of bringing NUMA nodes online and putting them
>> back offline, given that they require a device driver that may not yet have been
>> loaded. There are a few minor missing bits there.
>
> This has been prototyped with the driver doing memory hotplug/hotremove. Could you
> elaborate a little on what you feel is missing?
>
We just worked through how to deal with this in our driver, and I remember feeling
worried about the way NUMA nodes can only be put online via a user space action
(through sysfs). It seemed like you'd want to do that from kernel as well, when a
device driver gets loaded.
I was also uneasy about user space trying to bring a node online before the
associated device driver was loaded, and I think it would be nice to be sure that
that whole story is looked at.
The theme here is that driver load/unload is, today, independent from the NUMA node
online/offline, and that's a problem. Not a huge one, though, just worth enumerating
here.
thanks
john h
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [patch v2] mm, vmscan: avoid thrashing anon lru when free + file is low
From: David Rientjes @ 2017-05-01 21:34 UTC (permalink / raw)
To: Andrew Morton, Minchan Kim
Cc: Johannes Weiner, Mel Gorman, linux-kernel, linux-mm
In-Reply-To: <20170420060904.GA3720@bbox>
The purpose of the code that commit 623762517e23 ("revert 'mm: vmscan: do
not swap anon pages just because free+file is low'") reintroduces is to
prefer swapping anonymous memory rather than trashing the file lru.
If the anonymous inactive lru for the set of eligible zones is considered
low, however, or the length of the list for the given reclaim priority
does not allow for effective anonymous-only reclaiming, then avoid
forcing SCAN_ANON. Forcing SCAN_ANON will end up thrashing the small
list and leave unreclaimed memory on the file lrus.
If the inactive list is insufficient, fallback to balanced reclaim so the
file lru doesn't remain untouched.
Suggested-by: Minchan Kim <minchan@kernel.org>
Signed-off-by: David Rientjes <rientjes@google.com>
---
to akpm: this issue has been possible since at least 3.15, so it's
probably not high priority for 4.12 but applies cleanly if it can sneak
in
mm/vmscan.c | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/mm/vmscan.c b/mm/vmscan.c
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2204,8 +2204,17 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg,
}
if (unlikely(pgdatfile + pgdatfree <= total_high_wmark)) {
- scan_balance = SCAN_ANON;
- goto out;
+ /*
+ * Force SCAN_ANON if there are enough inactive
+ * anonymous pages on the LRU in eligible zones.
+ * Otherwise, the small LRU gets thrashed.
+ */
+ if (!inactive_list_is_low(lruvec, false, sc, false) &&
+ lruvec_lru_size(lruvec, LRU_INACTIVE_ANON, sc->reclaim_idx)
+ >> sc->priority) {
+ scan_balance = SCAN_ANON;
+ goto out;
+ }
}
}
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: Reza Arbab @ 2017-05-01 21:04 UTC (permalink / raw)
To: John Hubbard
Cc: Balbir Singh, linux-mm, akpm, khandual, benh, aneesh.kumar,
paulmck, srikar, haren, jglisse, mgorman, mhocko, vbabka, cl
In-Reply-To: <91272c14-81df-9529-f0ae-6abb17a694ea@nvidia.com>
On Mon, May 01, 2017 at 01:41:55PM -0700, John Hubbard wrote:
>1. A way to move pages between NUMA nodes, both virtual address and
>physical address-based, from kernel mode.
Jerome's migrate_vma() and migrate_dma() should have this covered,
including DMA-accelerated copy.
>5. Something to handle the story of bringing NUMA nodes online and
>putting them back offline, given that they require a device driver that
>may not yet have been loaded. There are a few minor missing bits there.
This has been prototyped with the driver doing memory hotplug/hotremove.
Could you elaborate a little on what you feel is missing?
--
Reza Arbab
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: John Hubbard @ 2017-05-01 20:41 UTC (permalink / raw)
To: Balbir Singh, linux-mm, akpm
Cc: khandual, benh, aneesh.kumar, paulmck, srikar, haren, jglisse,
mgorman, mhocko, arbab, vbabka, cl
In-Reply-To: <20170419075242.29929-1-bsingharora@gmail.com>
On 04/19/2017 12:52 AM, Balbir Singh wrote:
> This is a request for comments on the discussed approaches
> for coherent memory at mm-summit (some of the details are at
> https://lwn.net/Articles/717601/). The latest posted patch
> series is at https://lwn.net/Articles/713035/. I am reposting
> this as RFC, Michal Hocko suggested using HMM for CDM, but
> we believe there are stronger reasons to use the NUMA approach.
> The earlier patches for Coherent Device memory were implemented
> and designed by Anshuman Khandual.
>
Hi Balbir,
Although I think everyone agrees that in the [very] long term, these
hardware-coherent nodes probably want to be NUMA nodes, in order to decide what to
code up over the next few years, we need to get a clear idea of what has to be done
for each possible approach.
Here, the CDM discussion is falling just a bit short, because it does not yet
include the whole story of what we would need to do. Earlier threads pointed this
out: the idea started as a large patchset RFC, but then, "for ease of review", it
got turned into a smaller RFC, which loses too much context.
So, I'd suggest putting together something more complete, so that it can be fairly
compared against the HMM-for-hardware-coherent-nodes approach.
> Jerome posted HMM-CDM at https://lwn.net/Articles/713035/.
> The patches do a great deal to enable CDM with HMM, but we
> still believe that HMM with CDM is not a natural way to
> represent coherent device memory and the mm will need
> to be audited and enhanced for it to even work.
That is also true for the CDM approach. Specifically, in order for this to be of any
use to device drivers, we'll need the following:
1. A way to move pages between NUMA nodes, both virtual address and physical
address-based, from kernel mode.
2. A way to provide reverse mapping information to device drivers, even if
indirectly. (I'm not proposing exposing rmap, but this has to be thought through,
because at some point, a device will need to do something with a physical page.)
This strikes me as the hardest part of the problem.
3. Detection and mitigation of page thrashing between NUMA nodes (shared
responsibility between core -mm and device driver, but probably missing some APIs
today).
4. Handling of oversubscription (allocating more memory than is physically on a NUMA
node, by evicting "LRU-like" pages, rather than the current fallback to other NUMA
nodes). Similar to (3) with respect to where we're at today.
5. Something to handle the story of bringing NUMA nodes online and putting them back
offline, given that they require a device driver that may not yet have been loaded.
There are a few minor missing bits there.
thanks,
--
John Hubbard
NVIDIA
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2] mm, zone_device: replace {get, put}_zone_device_page() with a single reference
From: Jerome Glisse @ 2017-05-01 20:32 UTC (permalink / raw)
To: Dan Williams
Cc: Kirill A. Shutemov, Ingo Molnar, linux-kernel@vger.kernel.org,
Linux MM, Ingo Molnar, Andrew Morton, Logan Gunthorpe,
Kirill Shutemov
In-Reply-To: <CAPcyv4gFMyXhqY9enam5v9nFwjSULLE=PUEqGP0psLMcA9fzDA@mail.gmail.com>
On Mon, May 01, 2017 at 01:19:24PM -0700, Dan Williams wrote:
> On Mon, May 1, 2017 at 6:55 AM, Jerome Glisse <jglisse@redhat.com> wrote:
> > On Mon, May 01, 2017 at 01:23:59PM +0300, Kirill A. Shutemov wrote:
> >> On Sun, Apr 30, 2017 at 07:14:24PM -0400, Jerome Glisse wrote:
> >> > On Sat, Apr 29, 2017 at 01:17:26PM +0300, Kirill A. Shutemov wrote:
> >> > > On Fri, Apr 28, 2017 at 03:33:07PM -0400, Jerome Glisse wrote:
> >> > > > On Fri, Apr 28, 2017 at 12:22:24PM -0700, Dan Williams wrote:
> >> > > > > Are you sure about needing to hook the 2 -> 1 transition? Could we
> >> > > > > change ZONE_DEVICE pages to not have an elevated reference count when
> >> > > > > they are created so you can keep the HMM references out of the mm hot
> >> > > > > path?
> >> > > >
> >> > > > 100% sure on that :) I need to callback into driver for 2->1 transition
> >> > > > no way around that. If we change ZONE_DEVICE to not have an elevated
> >> > > > reference count that you need to make a lot more change to mm so that
> >> > > > ZONE_DEVICE is never use as fallback for memory allocation. Also need
> >> > > > to make change to be sure that ZONE_DEVICE page never endup in one of
> >> > > > the path that try to put them back on lru. There is a lot of place that
> >> > > > would need to be updated and it would be highly intrusive and add a
> >> > > > lot of special cases to other hot code path.
> >> > >
> >> > > Could you explain more on where the requirement comes from or point me to
> >> > > where I can read about this.
> >> > >
> >> >
> >> > HMM ZONE_DEVICE pages are use like other pages (anonymous or file back page)
> >> > in _any_ vma. So i need to know when a page is freed ie either as result of
> >> > unmap, exit or migration or anything that would free the memory. For zone
> >> > device a page is free once its refcount reach 1 so i need to catch refcount
> >> > transition from 2->1
> >>
> >> What if we would rework zone device to have pages with refcount 0 at
> >> start?
> >
> > That is a _lot_ of work from top of my head because it would need changes
> > to a lot of places and likely more hot code path that simply adding some-
> > thing to put_page() note that i only need something in put_page() i do not
> > need anything in the get page path. Is adding a conditional branch for
> > HMM pages in put_page() that much of a problem ?
> >
> >
> >> > This is the only way i can inform the device that the page is now free. See
> >> >
> >> > https://cgit.freedesktop.org/~glisse/linux/commit/?h=hmm-v21&id=52da8fe1a088b87b5321319add79e43b8372ed7d
> >> >
> >> > There is _no_ way around that.
> >>
> >> I'm still not convinced that it's impossible.
> >>
> >> Could you describe lifecycle for pages in case of HMM?
> >
> > Process malloc something, end it over to some function in the program
> > that use the GPU that function call GPU API (OpenCL, CUDA, ...) that
> > trigger a migration to device memory.
> >
> > So in the kernel you get a migration like any existing migration,
> > original page is unmap, if refcount is all ok (no pin) then a device
> > page is allocated and thing are migrated to device memory.
> >
> > What happen after is unknown. Either userspace/kernel driver decide
> > to migrate back to system memory, either there is an munmap, either
> > there is a CPU page fault, ... So from that point on the device page
> > as the exact same life as a regular page.
> >
> > Above i describe the migrate case, but you can also have new memory
> > allocation that directly allocate device memory. For instance if the
> > GPU do a page fault on an address that isn't back by anything then
> > we can directly allocate a device page. No migration involve in that
> > case.
> >
> > HMM pages are like any other pages in most respect. Exception are:
> > - no GUP
> > - no KSM
> > - no lru reclaim
> > - no NUMA balancing
> > - no regular migration (existing migrate_page)
> >
> > The fact that minimum refcount for ZONE_DEVICE is 1 already gives
> > us for free most of the above exception. To convert the refcount to
> > be like other pages would mean that all of the above would need to
> > be audited and probably modify to ignore ZONE_DEVICE pages (i am
> > pretty sure Dan do not want any of the above either).
>
> Right, adding HMM references to get_page() and put_page() seems less
> intrusive. Given how uncommon HMM hardware is (insert grumble about no
> visible upstream user of this functionality) I think the 'static
> branch' approach helps mitigate the impact for everything else.
> Looking back, I should have used that mechanism for the pmem use case,
> but it's moot now.
I do not need anything in get_page() all i need is something in put_page()
to catch the 2 -> 1 refcount transition to know when a page is freed.
Cheers,
Jerome
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2] mm, zone_device: replace {get, put}_zone_device_page() with a single reference
From: Dan Williams @ 2017-05-01 20:19 UTC (permalink / raw)
To: Jerome Glisse
Cc: Kirill A. Shutemov, Ingo Molnar, linux-kernel@vger.kernel.org,
Linux MM, Ingo Molnar, Andrew Morton, Logan Gunthorpe,
Kirill Shutemov
In-Reply-To: <20170501135545.GA16772@redhat.com>
On Mon, May 1, 2017 at 6:55 AM, Jerome Glisse <jglisse@redhat.com> wrote:
> On Mon, May 01, 2017 at 01:23:59PM +0300, Kirill A. Shutemov wrote:
>> On Sun, Apr 30, 2017 at 07:14:24PM -0400, Jerome Glisse wrote:
>> > On Sat, Apr 29, 2017 at 01:17:26PM +0300, Kirill A. Shutemov wrote:
>> > > On Fri, Apr 28, 2017 at 03:33:07PM -0400, Jerome Glisse wrote:
>> > > > On Fri, Apr 28, 2017 at 12:22:24PM -0700, Dan Williams wrote:
>> > > > > Are you sure about needing to hook the 2 -> 1 transition? Could we
>> > > > > change ZONE_DEVICE pages to not have an elevated reference count when
>> > > > > they are created so you can keep the HMM references out of the mm hot
>> > > > > path?
>> > > >
>> > > > 100% sure on that :) I need to callback into driver for 2->1 transition
>> > > > no way around that. If we change ZONE_DEVICE to not have an elevated
>> > > > reference count that you need to make a lot more change to mm so that
>> > > > ZONE_DEVICE is never use as fallback for memory allocation. Also need
>> > > > to make change to be sure that ZONE_DEVICE page never endup in one of
>> > > > the path that try to put them back on lru. There is a lot of place that
>> > > > would need to be updated and it would be highly intrusive and add a
>> > > > lot of special cases to other hot code path.
>> > >
>> > > Could you explain more on where the requirement comes from or point me to
>> > > where I can read about this.
>> > >
>> >
>> > HMM ZONE_DEVICE pages are use like other pages (anonymous or file back page)
>> > in _any_ vma. So i need to know when a page is freed ie either as result of
>> > unmap, exit or migration or anything that would free the memory. For zone
>> > device a page is free once its refcount reach 1 so i need to catch refcount
>> > transition from 2->1
>>
>> What if we would rework zone device to have pages with refcount 0 at
>> start?
>
> That is a _lot_ of work from top of my head because it would need changes
> to a lot of places and likely more hot code path that simply adding some-
> thing to put_page() note that i only need something in put_page() i do not
> need anything in the get page path. Is adding a conditional branch for
> HMM pages in put_page() that much of a problem ?
>
>
>> > This is the only way i can inform the device that the page is now free. See
>> >
>> > https://cgit.freedesktop.org/~glisse/linux/commit/?h=hmm-v21&id=52da8fe1a088b87b5321319add79e43b8372ed7d
>> >
>> > There is _no_ way around that.
>>
>> I'm still not convinced that it's impossible.
>>
>> Could you describe lifecycle for pages in case of HMM?
>
> Process malloc something, end it over to some function in the program
> that use the GPU that function call GPU API (OpenCL, CUDA, ...) that
> trigger a migration to device memory.
>
> So in the kernel you get a migration like any existing migration,
> original page is unmap, if refcount is all ok (no pin) then a device
> page is allocated and thing are migrated to device memory.
>
> What happen after is unknown. Either userspace/kernel driver decide
> to migrate back to system memory, either there is an munmap, either
> there is a CPU page fault, ... So from that point on the device page
> as the exact same life as a regular page.
>
> Above i describe the migrate case, but you can also have new memory
> allocation that directly allocate device memory. For instance if the
> GPU do a page fault on an address that isn't back by anything then
> we can directly allocate a device page. No migration involve in that
> case.
>
> HMM pages are like any other pages in most respect. Exception are:
> - no GUP
> - no KSM
> - no lru reclaim
> - no NUMA balancing
> - no regular migration (existing migrate_page)
>
> The fact that minimum refcount for ZONE_DEVICE is 1 already gives
> us for free most of the above exception. To convert the refcount to
> be like other pages would mean that all of the above would need to
> be audited and probably modify to ignore ZONE_DEVICE pages (i am
> pretty sure Dan do not want any of the above either).
Right, adding HMM references to get_page() and put_page() seems less
intrusive. Given how uncommon HMM hardware is (insert grumble about no
visible upstream user of this functionality) I think the 'static
branch' approach helps mitigate the impact for everything else.
Looking back, I should have used that mechanism for the pmem use case,
but it's moot now.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH man-pages 1/2] userfaultfd.2: start documenting non-cooperative events
From: Michael Kerrisk (man-pages) @ 2017-05-01 18:34 UTC (permalink / raw)
To: Mike Rapoprt
Cc: mtk.manpages, Andrea Arcangeli, linux-kernel, linux-mm, linux-man
In-Reply-To: <190E3CFC-492F-4672-9385-9C3D8F57F26C@linux.vnet.ibm.com>
Hi Mike,
On 04/28/2017 11:45 AM, Mike Rapoprt wrote:
>
>
> On April 27, 2017 8:26:16 PM GMT+03:00, "Michael Kerrisk (man-pages)" <mtk.manpages@gmail.com> wrote:
>> Hi Mike,
>>
>> I've applied this, but have some questions/points I think
>> further clarification.
>>
>> On 04/27/2017 04:14 PM, Mike Rapoport wrote:
>>> Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
>>> ---
>>> man2/userfaultfd.2 | 135
>> ++++++++++++++++++++++++++++++++++++++++++++++++++---
>>> 1 file changed, 128 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
>>> index cfea5cb..44af3e4 100644
>>> --- a/man2/userfaultfd.2
>>> +++ b/man2/userfaultfd.2
>>> @@ -75,7 +75,7 @@ flag in
>>> .PP
>>> When the last file descriptor referring to a userfaultfd object is
>> closed,
>>> all memory ranges that were registered with the object are
>> unregistered
>>> -and unread page-fault events are flushed.
>>> +and unread events are flushed.
>>> .\"
>>> .SS Usage
>>> The userfaultfd mechanism is designed to allow a thread in a
>> multithreaded
>>> @@ -99,6 +99,20 @@ In such non-cooperative mode,
>>> the process that monitors userfaultfd and handles page faults
>>> needs to be aware of the changes in the virtual memory layout
>>> of the faulting process to avoid memory corruption.
>>> +
>>> +Starting from Linux 4.11,
>>> +userfaultfd may notify the fault-handling threads about changes
>>> +in the virtual memory layout of the faulting process.
>>> +In addition, if the faulting process invokes
>>> +.BR fork (2)
>>> +system call,
>>> +the userfaultfd objects associated with the parent may be duplicated
>>> +into the child process and the userfaultfd monitor will be notified
>>> +about the file descriptor associated with the userfault objects
>>
>> What does "notified about the file descriptor" mean?
>
> Well, seems that I've made this one really awkward :)
> When the monitored process forks, all the userfault objects
> associateda?? with it are duplicated into the child process. For each
> duplicated object, userfault generates event of type UFFD_EVENT_FORK
> and the uffdio_msg for this event contains the file descriptor that
> should be used to manipulate the duplicated userfault object.
> Hope this clarifies.
Yes, it's clearer now.
Mostly what was needed here was a forward reference that mentions
UFFD_EVENT_FORK explicitly. I added that, and also enhanced the
text on UFFD_EVENT_FORK a little.
Also, it's not just fork(2) for which UFFD_EVENT_FORK is generated,
right? It can also be a clone(2) cal that does not specify
CLONE_VM, right?
Could you review my changes in commit 522ab2ff6fc9010432a
to make sure they are okay.
Cheers,
Michael
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH man-pages 0/5] {ioctl_}userfaultfd.2: yet another update
From: Michael Kerrisk (man-pages) @ 2017-05-01 18:34 UTC (permalink / raw)
To: Mike Rapoport
Cc: mtk.manpages, Andrea Arcangeli, linux-kernel, linux-mm, linux-man
In-Reply-To: <1493617399-20897-1-git-send-email-rppt@linux.vnet.ibm.com>
Hi Mike,
On 05/01/2017 07:43 AM, Mike Rapoport wrote:
> Hi Michael,
>
> These updates pretty much complete the coverage of 4.11 additions, IMHO.
Thanks for this, but we still await input from Andrea
on various points.
> Mike Rapoport (5):
> ioctl_userfaultfd.2: update description of shared memory areas
> ioctl_userfaultfd.2: UFFDIO_COPY: add ENOENT and ENOSPC description
> ioctl_userfaultfd.2: add BUGS section
> userfaultfd.2: add note about asynchronios events delivery
> userfaultfd.2: update VERSIONS section with 4.11 chanegs
>
> man2/ioctl_userfaultfd.2 | 35 +++++++++++++++++++++++++++++++++--
> man2/userfaultfd.2 | 15 +++++++++++++++
> 2 files changed, 48 insertions(+), 2 deletions(-)
I've applied all of the above, and done some light editing.
Could you please check my changes in the following commits:
5191c68806c8ac73fdc89586cde434d2766abb5c
265225c1e2311ae26ead116e6c8d2cedd46144fa
Thanks,
Michael
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH man-pages 5/5] userfaultfd.2: update VERSIONS section with 4.11 chanegs
From: Michael Kerrisk (man-pages) @ 2017-05-01 18:33 UTC (permalink / raw)
To: Mike Rapoport
Cc: mtk.manpages, Andrea Arcangeli, linux-kernel, linux-mm, linux-man
In-Reply-To: <1493617399-20897-6-git-send-email-rppt@linux.vnet.ibm.com>
Hi Mike,
On 05/01/2017 07:43 AM, Mike Rapoport wrote:
> Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Thanks. Applied.
Cheers,
Michael
> ---
> man2/userfaultfd.2 | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
> index f177bba..07a69f1 100644
> --- a/man2/userfaultfd.2
> +++ b/man2/userfaultfd.2
> @@ -404,6 +404,9 @@ Insufficient kernel memory was available.
> The
> .BR userfaultfd ()
> system call first appeared in Linux 4.3.
> +
> +The support for hugetlbfs and shared memory areas and
> +non-page-fault events was added in Linux 4.11
> .SH CONFORMING TO
> .BR userfaultfd ()
> is Linux-specific and should not be used in programs intended to be
>
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH man-pages 3/5] ioctl_userfaultfd.2: add BUGS section
From: Michael Kerrisk (man-pages) @ 2017-05-01 18:33 UTC (permalink / raw)
To: Mike Rapoport
Cc: mtk.manpages, Andrea Arcangeli, linux-kernel, linux-mm, linux-man
In-Reply-To: <1493617399-20897-4-git-send-email-rppt@linux.vnet.ibm.com>
Hi Mike,
I've applied this, but have a question.
On 05/01/2017 07:43 AM, Mike Rapoport wrote:
> The features handshake is not quite convenient.
> Elaborate about it in the BUGS section.
>
> Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
> ---
> man2/ioctl_userfaultfd.2 | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
> index e12b9de..50316de 100644
> --- a/man2/ioctl_userfaultfd.2
> +++ b/man2/ioctl_userfaultfd.2
> @@ -650,6 +650,15 @@ operations are Linux-specific.
> .SH EXAMPLE
> See
> .BR userfaultfd (2).
> +.SH BUGS
> +In order to detect available userfault features and
> +enable certain subset of those features
I changed "certain" to "some". ("certain subset" here also
would sound like "some particular subset" of those features.)
Okay?
> +the usefault file descriptor must be closed after the first
> +.BR UFFDIO_API
> +operation that queries features availability and re-opened before
> +the second
> +.BR UFFDIO_API
> +call that actually enables the desired features.
> .SH SEE ALSO
> .BR ioctl (2),
> .BR mmap (2),
Cheers,
Michael
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH man-pages 4/5] userfaultfd.2: add note about asynchronios events delivery
From: Michael Kerrisk (man-pages) @ 2017-05-01 18:33 UTC (permalink / raw)
To: Mike Rapoport
Cc: mtk.manpages, Andrea Arcangeli, linux-kernel, linux-mm, linux-man
In-Reply-To: <1493617399-20897-5-git-send-email-rppt@linux.vnet.ibm.com>
Hi Mike,
On 05/01/2017 07:43 AM, Mike Rapoport wrote:
> Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Thanks. Applied. One question below.
> ---
> man2/userfaultfd.2 | 12 ++++++++++++
> 1 file changed, 12 insertions(+)
>
> diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
> index 8b89162..f177bba 100644
> --- a/man2/userfaultfd.2
> +++ b/man2/userfaultfd.2
> @@ -112,6 +112,18 @@ created for the child process,
> which allows userfaultfd monitor to perform user-space paging
> for the child process.
>
> +Unlike page faults which have to be synchronous and require
> +explicit or implicit wakeup,
> +all other events are delivered asynchronously and
> +the non-cooperative process resumes execution as
> +soon as manager executes
> +.BR read(2).
> +The userfaultfd manager should carefully synchronize calls
> +to UFFDIO_COPY with the events processing.
> +
> +The current asynchronous model of the event delivery is optimal for
> +single threaded non-cooperative userfaultfd manager implementations.
The preceding paragraph feels incomplete. It seems like you want to make
a point with that last sentence, but the point is not explicit. What's
missing?
> +
> .\" FIXME elaborate about non-cooperating mode, describe its limitations
> .\" for kernels before 4.11, features added in 4.11
> .\" and limitations remaining in 4.11
>
Cheers,
Michael
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH man-pages 2/5] ioctl_userfaultfd.2: UFFDIO_COPY: add ENOENT and ENOSPC description
From: Michael Kerrisk (man-pages) @ 2017-05-01 18:33 UTC (permalink / raw)
To: Mike Rapoport
Cc: mtk.manpages, Andrea Arcangeli, linux-kernel, linux-mm, linux-man
In-Reply-To: <1493617399-20897-3-git-send-email-rppt@linux.vnet.ibm.com>
On 05/01/2017 07:43 AM, Mike Rapoport wrote:
> Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Thanks, Mike. Applied.
Cheers,
Michael
> ---
> man2/ioctl_userfaultfd.2 | 13 +++++++++++++
> 1 file changed, 13 insertions(+)
>
> diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
> index 6edd396..e12b9de 100644
> --- a/man2/ioctl_userfaultfd.2
> +++ b/man2/ioctl_userfaultfd.2
> @@ -481,6 +481,19 @@ was invalid.
> An invalid bit was specified in the
> .IR mode
> field.
> +.TP
> +.B ENOENT
> +(Since Linux 4.11)
> +The faulting process has changed
> +its virtual memory layout simultaneously with outstanding
> +.I UFFDIO_COPY
> +operation.
> +.TP
> +.B ENOSPC
> +(Since Linux 4.11)
> +The faulting process has exited at the time of
> +.I UFFDIO_COPY
> +operation.
> .\"
> .SS UFFDIO_ZEROPAGE
> (Since Linux 4.3.)
>
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH man-pages 1/5] ioctl_userfaultfd.2: update description of shared memory areas
From: Michael Kerrisk (man-pages) @ 2017-05-01 18:33 UTC (permalink / raw)
To: Mike Rapoport
Cc: mtk.manpages, Andrea Arcangeli, linux-kernel, linux-mm, linux-man
In-Reply-To: <1493617399-20897-2-git-send-email-rppt@linux.vnet.ibm.com>
Hello Mike,
I've applied this patch, but have a question.
On 05/01/2017 07:43 AM, Mike Rapoport wrote:
> Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
> ---
> man2/ioctl_userfaultfd.2 | 13 +++++++++++--
> 1 file changed, 11 insertions(+), 2 deletions(-)
>
> diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
> index 889feb9..6edd396 100644
> --- a/man2/ioctl_userfaultfd.2
> +++ b/man2/ioctl_userfaultfd.2
> @@ -181,8 +181,17 @@ virtual memory areas
> .TP
> .B UFFD_FEATURE_MISSING_SHMEM
> If this feature bit is set,
> -the kernel supports registering userfaultfd ranges on tmpfs
> -virtual memory areas
> +the kernel supports registering userfaultfd ranges on shared memory areas.
> +This includes all kernel shared memory APIs:
> +System V shared memory,
> +tmpfs,
> +/dev/zero,
> +.BR mmap(2)
> +with
> +.I MAP_SHARED
> +flag set,
> +.BR memfd_create (2),
> +etc.
>
> The returned
> .I ioctls
Does the change in this patch represent a change that occurred in
Linux 4.11? If so, I think this needs to be said explicitly in the text.
Cheers,
Michael
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH RFC] hugetlbfs 'noautofill' mount option
From: Prakash Sangappa @ 2017-05-01 18:00 UTC (permalink / raw)
To: linux-kernel, linux-mm
In-Reply-To: <326e38dd-b4a8-e0ca-6ff7-af60e8045c74@oracle.com>
Some applications like a database use hugetblfs for performance
reasons. Files on hugetlbfs filesystem are created and huge pages
allocated using fallocate() API. Pages are deallocated/freed using
fallocate() hole punching support that has been added to hugetlbfs.
These files are mmapped and accessed by many processes as shared memory.
Such applications keep track of which offsets in the hugetlbfs file have
pages allocated.
Any access to mapped address over holes in the file, which can occur due
to bugs in the application, is considered invalid and expect the process
to simply receive a SIGBUS. However, currently when a hole in the file is
accessed via the mapped address, kernel/mm attempts to automatically
allocate a page at page fault time, resulting in implicitly filling the hole
in the file. This may not be the desired behavior for applications like the
database that want to explicitly manage page allocations of hugetlbfs files.
This patch adds a new hugetlbfs mount option 'noautofill', to indicate that
pages should not be allocated at page fault time when accessed thru mmapped
address.
Signed-off-by: Prakash <prakash.sangappa@oracle.com>
---
fs/hugetlbfs/inode.c | 11 ++++++++++-
include/linux/hugetlb.h | 1 +
mm/hugetlb.c | 5 +++++
3 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index 8f96461..8342ee9 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -53,6 +53,7 @@ struct hugetlbfs_config {
long nr_inodes;
struct hstate *hstate;
long min_hpages;
+ long noautofill;
};
struct hugetlbfs_inode_info {
@@ -71,7 +72,7 @@ enum {
Opt_size, Opt_nr_inodes,
Opt_mode, Opt_uid, Opt_gid,
Opt_pagesize, Opt_min_size,
- Opt_err,
+ Opt_noautofill, Opt_err,
};
static const match_table_t tokens = {
@@ -82,6 +83,7 @@ static const match_table_t tokens = {
{Opt_gid, "gid=%u"},
{Opt_pagesize, "pagesize=%s"},
{Opt_min_size, "min_size=%s"},
+ {Opt_noautofill, "noautofill"},
{Opt_err, NULL},
};
@@ -1109,6 +1111,11 @@ hugetlbfs_parse_options(char *options, struct
hugetlbfs_config *pconfig)
break;
}
+ case Opt_noautofill: {
+ pconfig->noautofill = 1;
+ break;
+ }
+
default:
pr_err("Bad mount option: \"%s\"\n", p);
return -EINVAL;
@@ -1157,6 +1164,7 @@ hugetlbfs_fill_super(struct super_block *sb, void
*data, int silent)
config.mode = 0755;
config.hstate = &default_hstate;
config.min_hpages = -1; /* No default minimum size */
+ config.noautofill = 0;
ret = hugetlbfs_parse_options(data, &config);
if (ret)
return ret;
@@ -1170,6 +1178,7 @@ hugetlbfs_fill_super(struct super_block *sb, void
*data, int silent)
sbinfo->max_inodes = config.nr_inodes;
sbinfo->free_inodes = config.nr_inodes;
sbinfo->spool = NULL;
+ sbinfo->noautofill = config.noautofill;
/*
* Allocate and initialize subpool if maximum or minimum size is
* specified. Any needed reservations (for minimim size) are taken
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 503099d..2f37e0c 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -259,6 +259,7 @@ struct hugetlbfs_sb_info {
spinlock_t stat_lock;
struct hstate *hstate;
struct hugepage_subpool *spool;
+ int noautofill; /* don't allocate page to fill hole at fault time */
};
static inline struct hugetlbfs_sb_info *HUGETLBFS_SB(struct
super_block *sb)
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index a7aa811..11655ef 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -3715,6 +3715,11 @@ static int hugetlb_no_page(struct mm_struct *mm,
struct vm_area_struct *vma,
goto out;
}
+ if (HUGETLBFS_SB(mapping->host->i_sb)->noautofill) {
+ ret = VM_FAULT_SIGBUS;
+ goto out;
+ }
+
page = alloc_huge_page(vma, address, 0);
if (IS_ERR(page)) {
ret = PTR_ERR(page);
--
2.7.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [PATCH 1/2] dax: prevent invalidation of mapped DAX entries
From: Ross Zwisler @ 2017-05-01 16:54 UTC (permalink / raw)
To: Jan Kara
Cc: Ross Zwisler, Andrew Morton, linux-kernel, Alexander Viro,
Alexey Kuznetsov, Andrey Ryabinin, Anna Schumaker,
Christoph Hellwig, Dan Williams, Darrick J. Wong,
Eric Van Hensbergen, Jens Axboe, Johannes Weiner,
Konrad Rzeszutek Wilk, Latchesar Ionkov, linux-cifs,
linux-fsdevel, linux-mm, linux-nfs, linux-nvdimm, Matthew Wilcox,
Ron Minnich, samba-technical, Steve French, Trond Myklebust,
v9fs-developer
In-Reply-To: <20170425101041.GG2793@quack2.suse.cz>
On Tue, Apr 25, 2017 at 12:10:41PM +0200, Jan Kara wrote:
> On Thu 20-04-17 21:44:36, Ross Zwisler wrote:
> > dax_invalidate_mapping_entry() currently removes DAX exceptional entries
> > only if they are clean and unlocked. This is done via:
> >
> > invalidate_mapping_pages()
> > invalidate_exceptional_entry()
> > dax_invalidate_mapping_entry()
> >
> > However, for page cache pages removed in invalidate_mapping_pages() there
> > is an additional criteria which is that the page must not be mapped. This
> > is noted in the comments above invalidate_mapping_pages() and is checked in
> > invalidate_inode_page().
> >
> > For DAX entries this means that we can can end up in a situation where a
> > DAX exceptional entry, either a huge zero page or a regular DAX entry,
> > could end up mapped but without an associated radix tree entry. This is
> > inconsistent with the rest of the DAX code and with what happens in the
> > page cache case.
> >
> > We aren't able to unmap the DAX exceptional entry because according to its
> > comments invalidate_mapping_pages() isn't allowed to block, and
> > unmap_mapping_range() takes a write lock on the mapping->i_mmap_rwsem.
> >
> > Since we essentially never have unmapped DAX entries to evict from the
> > radix tree, just remove dax_invalidate_mapping_entry().
> >
> > Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
> > Fixes: c6dcf52c23d2 ("mm: Invalidate DAX radix tree entries only if appropriate")
> > Reported-by: Jan Kara <jack@suse.cz>
> > Cc: <stable@vger.kernel.org> [4.10+]
>
> Just as a side note - we wouldn't really have to unmap the mapping range
> covered by the DAX exceptional entry. It would be enough to find out
> whether such range is mapped and bail out in that case. But that would
> still be pretty expensive for DAX - we'd have to do rmap walk similar as in
> dax_mapping_entry_mkclean() and IMHO it is not worth it. So I agree with
> what you did. You can add:
>
> Reviewed-by: Jan Kara <jack@suse.cz>
Yep, that makes sense. Thanks for the review.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox