* Re: [PATCH v2] mm, zone_device: replace {get, put}_zone_device_page() with a single reference
From: Logan Gunthorpe @ 2017-05-01 3:48 UTC (permalink / raw)
To: Jerome Glisse, Kirill A. Shutemov
Cc: Dan Williams, Ingo Molnar, linux-kernel@vger.kernel.org, Linux MM,
Ingo Molnar, Andrew Morton, Kirill Shutemov
In-Reply-To: <20170430231421.GA15163@redhat.com>
On 30/04/17 05:14 PM, Jerome Glisse wrote:
> HMM ZONE_DEVICE pages are use like other pages (anonymous or file back page)
> in _any_ vma. So i need to know when a page is freed ie either as result of
> unmap, exit or migration or anything that would free the memory. For zone
> device a page is free once its refcount reach 1 so i need to catch refcount
> transition from 2->1
>
> This is the only way i can inform the device that the page is now free. See
>
> https://cgit.freedesktop.org/~glisse/linux/commit/?h=hmm-v21&id=52da8fe1a088b87b5321319add79e43b8372ed7d
>
> There is _no_ way around that.
I had a similar issue in a piece of my p2pmem RFC [1]. I hacked around
it by tracking the pages separately and freeing them when the vma is
closed. This is by no means a great solution, it certainly has it's own
warts. However, maybe it will spark some ideas for some alternate
choices which avoid the hot path.
Another thing I briefly looked at was hooking the vma close process
earlier so that it would callback in time that you can loop through the
pages and do your free process. Of course this all depends on the vma
not getting closed while the pages have other references. So it may not
work at all. Again, just ideas.
Logan
[1]
https://github.com/sbates130272/linux-p2pmem/commit/77c631d92cb5c451c9824b3a4cf9b6cddfde6bb7
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH man-pages 0/5] {ioctl_}userfaultfd.2: yet another update
From: Mike Rapoport @ 2017-05-01 5:43 UTC (permalink / raw)
To: Michael Kerrisk (man-pages)
Cc: Andrea Arcangeli, linux-kernel, linux-mm, linux-man,
Mike Rapoport
Hi Michael,
These updates pretty much complete the coverage of 4.11 additions, IMHO.
Mike Rapoport (5):
ioctl_userfaultfd.2: update description of shared memory areas
ioctl_userfaultfd.2: UFFDIO_COPY: add ENOENT and ENOSPC description
ioctl_userfaultfd.2: add BUGS section
userfaultfd.2: add note about asynchronios events delivery
userfaultfd.2: update VERSIONS section with 4.11 chanegs
man2/ioctl_userfaultfd.2 | 35 +++++++++++++++++++++++++++++++++--
man2/userfaultfd.2 | 15 +++++++++++++++
2 files changed, 48 insertions(+), 2 deletions(-)
--
1.9.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH man-pages 1/5] ioctl_userfaultfd.2: update description of shared memory areas
From: Mike Rapoport @ 2017-05-01 5:43 UTC (permalink / raw)
To: Michael Kerrisk (man-pages)
Cc: Andrea Arcangeli, linux-kernel, linux-mm, linux-man,
Mike Rapoport
In-Reply-To: <1493617399-20897-1-git-send-email-rppt@linux.vnet.ibm.com>
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
---
man2/ioctl_userfaultfd.2 | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
index 889feb9..6edd396 100644
--- a/man2/ioctl_userfaultfd.2
+++ b/man2/ioctl_userfaultfd.2
@@ -181,8 +181,17 @@ virtual memory areas
.TP
.B UFFD_FEATURE_MISSING_SHMEM
If this feature bit is set,
-the kernel supports registering userfaultfd ranges on tmpfs
-virtual memory areas
+the kernel supports registering userfaultfd ranges on shared memory areas.
+This includes all kernel shared memory APIs:
+System V shared memory,
+tmpfs,
+/dev/zero,
+.BR mmap(2)
+with
+.I MAP_SHARED
+flag set,
+.BR memfd_create (2),
+etc.
The returned
.I ioctls
--
1.9.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH man-pages 2/5] ioctl_userfaultfd.2: UFFDIO_COPY: add ENOENT and ENOSPC description
From: Mike Rapoport @ 2017-05-01 5:43 UTC (permalink / raw)
To: Michael Kerrisk (man-pages)
Cc: Andrea Arcangeli, linux-kernel, linux-mm, linux-man,
Mike Rapoport
In-Reply-To: <1493617399-20897-1-git-send-email-rppt@linux.vnet.ibm.com>
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
---
man2/ioctl_userfaultfd.2 | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
index 6edd396..e12b9de 100644
--- a/man2/ioctl_userfaultfd.2
+++ b/man2/ioctl_userfaultfd.2
@@ -481,6 +481,19 @@ was invalid.
An invalid bit was specified in the
.IR mode
field.
+.TP
+.B ENOENT
+(Since Linux 4.11)
+The faulting process has changed
+its virtual memory layout simultaneously with outstanding
+.I UFFDIO_COPY
+operation.
+.TP
+.B ENOSPC
+(Since Linux 4.11)
+The faulting process has exited at the time of
+.I UFFDIO_COPY
+operation.
.\"
.SS UFFDIO_ZEROPAGE
(Since Linux 4.3.)
--
1.9.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH man-pages 3/5] ioctl_userfaultfd.2: add BUGS section
From: Mike Rapoport @ 2017-05-01 5:43 UTC (permalink / raw)
To: Michael Kerrisk (man-pages)
Cc: Andrea Arcangeli, linux-kernel, linux-mm, linux-man,
Mike Rapoport
In-Reply-To: <1493617399-20897-1-git-send-email-rppt@linux.vnet.ibm.com>
The features handshake is not quite convenient.
Elaborate about it in the BUGS section.
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
---
man2/ioctl_userfaultfd.2 | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
index e12b9de..50316de 100644
--- a/man2/ioctl_userfaultfd.2
+++ b/man2/ioctl_userfaultfd.2
@@ -650,6 +650,15 @@ operations are Linux-specific.
.SH EXAMPLE
See
.BR userfaultfd (2).
+.SH BUGS
+In order to detect available userfault features and
+enable certain subset of those features
+the usefault file descriptor must be closed after the first
+.BR UFFDIO_API
+operation that queries features availability and re-opened before
+the second
+.BR UFFDIO_API
+call that actually enables the desired features.
.SH SEE ALSO
.BR ioctl (2),
.BR mmap (2),
--
1.9.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH man-pages 4/5] userfaultfd.2: add note about asynchronios events delivery
From: Mike Rapoport @ 2017-05-01 5:43 UTC (permalink / raw)
To: Michael Kerrisk (man-pages)
Cc: Andrea Arcangeli, linux-kernel, linux-mm, linux-man,
Mike Rapoport
In-Reply-To: <1493617399-20897-1-git-send-email-rppt@linux.vnet.ibm.com>
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
---
man2/userfaultfd.2 | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
index 8b89162..f177bba 100644
--- a/man2/userfaultfd.2
+++ b/man2/userfaultfd.2
@@ -112,6 +112,18 @@ created for the child process,
which allows userfaultfd monitor to perform user-space paging
for the child process.
+Unlike page faults which have to be synchronous and require
+explicit or implicit wakeup,
+all other events are delivered asynchronously and
+the non-cooperative process resumes execution as
+soon as manager executes
+.BR read(2).
+The userfaultfd manager should carefully synchronize calls
+to UFFDIO_COPY with the events processing.
+
+The current asynchronous model of the event delivery is optimal for
+single threaded non-cooperative userfaultfd manager implementations.
+
.\" FIXME elaborate about non-cooperating mode, describe its limitations
.\" for kernels before 4.11, features added in 4.11
.\" and limitations remaining in 4.11
--
1.9.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH man-pages 5/5] userfaultfd.2: update VERSIONS section with 4.11 chanegs
From: Mike Rapoport @ 2017-05-01 5:43 UTC (permalink / raw)
To: Michael Kerrisk (man-pages)
Cc: Andrea Arcangeli, linux-kernel, linux-mm, linux-man,
Mike Rapoport
In-Reply-To: <1493617399-20897-1-git-send-email-rppt@linux.vnet.ibm.com>
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
---
man2/userfaultfd.2 | 3 +++
1 file changed, 3 insertions(+)
diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
index f177bba..07a69f1 100644
--- a/man2/userfaultfd.2
+++ b/man2/userfaultfd.2
@@ -404,6 +404,9 @@ Insufficient kernel memory was available.
The
.BR userfaultfd ()
system call first appeared in Linux 4.3.
+
+The support for hugetlbfs and shared memory areas and
+non-page-fault events was added in Linux 4.11
.SH CONFORMING TO
.BR userfaultfd ()
is Linux-specific and should not be used in programs intended to be
--
1.9.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 0/3] Implement page table accounting for powerpc
From: Balbir Singh @ 2017-05-01 6:34 UTC (permalink / raw)
To: vdavydov, mpe, oss; +Cc: linuxppc-dev, linux-mm, Balbir Singh
(3e79ec7 arch: x86: charge page tables to kmemcg) added support for page
table accounting). This patch is the second iteration to add
support, in the earlier iteration only book3s 64 bit was supported.
This iteration adds support for booke/3s/32 and 64 bit.
There is some ugliness in this patchset, pgalloc.h is included
from book3s_64_mmu_radix.c to reuse the pte/pmd/pud and pgd
management routines. We use #ifdef MODULE to provide a version
that provides full accounting. The alternatives are discussed
in patch 1 below
Changelog v2:
- Added support for 32 bit and booke
- Added hugepte alloc accounting
Balbir Singh (3):
powerpc/mm/book(e)(3s)/64: Add page table accounting
powerpc/mm/book(e)(3s)/32: Add page table accounting
powerpc/mm/hugetlb: Add support for page accounting
arch/powerpc/include/asm/book3s/32/pgalloc.h | 3 ++-
arch/powerpc/include/asm/book3s/64/pgalloc.h | 17 +++++++++++------
arch/powerpc/include/asm/nohash/32/pgalloc.h | 3 ++-
arch/powerpc/include/asm/nohash/64/pgalloc.h | 12 ++++++++----
arch/powerpc/include/asm/pgalloc.h | 14 ++++++++++++++
arch/powerpc/mm/hugetlbpage.c | 2 +-
arch/powerpc/mm/pgtable_32.c | 2 +-
arch/powerpc/mm/pgtable_64.c | 3 ++-
8 files changed, 41 insertions(+), 15 deletions(-)
--
2.9.3
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH v2 1/3] powerpc/mm/book(e)(3s)/64: Add page table accounting
From: Balbir Singh @ 2017-05-01 6:34 UTC (permalink / raw)
To: vdavydov, mpe, oss; +Cc: linuxppc-dev, linux-mm, Balbir Singh
In-Reply-To: <20170501063438.25237-1-bsingharora@gmail.com>
Introduce a helper pgtable_gfp_flags() which
just returns the current gfp flags and adds
__GFP_ACCOUNT to account for page table allocation.
The generic helper is added to include/asm/pgalloc.h
and has two variants - WARNING ugly bits ahead
1. If the header is included from a module, no check
for mm == &init_mm is done, since init_mm is not
exported
2. For kernel includes, the check is done and required
see (3e79ec7 arch: x86: charge page tables to kmemcg)
The fundamental assumption is that no module should be
doing pgd/pud/pmd and pte alloc's on behalf of init_mm
directly.
NOTE: This adds an overhead to pmd/pud/pgd allocations
similar to x86. The other alternative was to implement
pmd_alloc_kernel/pud_alloc_kernel and pgd_alloc_kernel
with their offset variants.
For 4k page size, pte_alloc_one no longer calls
pte_alloc_one_kernel.
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
arch/powerpc/include/asm/book3s/32/pgalloc.h | 3 ++-
arch/powerpc/include/asm/book3s/64/pgalloc.h | 17 +++++++++++------
arch/powerpc/include/asm/nohash/64/pgalloc.h | 12 ++++++++----
arch/powerpc/include/asm/pgalloc.h | 14 ++++++++++++++
arch/powerpc/mm/pgtable_64.c | 3 ++-
5 files changed, 37 insertions(+), 12 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/32/pgalloc.h b/arch/powerpc/include/asm/book3s/32/pgalloc.h
index d310546..a120e7f 100644
--- a/arch/powerpc/include/asm/book3s/32/pgalloc.h
+++ b/arch/powerpc/include/asm/book3s/32/pgalloc.h
@@ -31,7 +31,8 @@ extern struct kmem_cache *pgtable_cache[];
static inline pgd_t *pgd_alloc(struct mm_struct *mm)
{
- return kmem_cache_alloc(PGT_CACHE(PGD_INDEX_SIZE), GFP_KERNEL);
+ return kmem_cache_alloc(PGT_CACHE(PGD_INDEX_SIZE),
+ pgtable_gfp_flags(mm, GFP_KERNEL));
}
static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
diff --git a/arch/powerpc/include/asm/book3s/64/pgalloc.h b/arch/powerpc/include/asm/book3s/64/pgalloc.h
index cd5e7aa..9aa904f 100644
--- a/arch/powerpc/include/asm/book3s/64/pgalloc.h
+++ b/arch/powerpc/include/asm/book3s/64/pgalloc.h
@@ -53,10 +53,11 @@ extern void __tlb_remove_table(void *_table);
static inline pgd_t *radix__pgd_alloc(struct mm_struct *mm)
{
#ifdef CONFIG_PPC_64K_PAGES
- return (pgd_t *)__get_free_page(PGALLOC_GFP);
+ return (pgd_t *)__get_free_page(pgtable_gfp_flags(mm, PGALLOC_GFP));
#else
struct page *page;
- page = alloc_pages(PGALLOC_GFP | __GFP_REPEAT, 4);
+ page = alloc_pages(pgtable_gfp_flags(mm, PGALLOC_GFP | __GFP_REPEAT),
+ 4);
if (!page)
return NULL;
return (pgd_t *) page_address(page);
@@ -76,7 +77,8 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm)
{
if (radix_enabled())
return radix__pgd_alloc(mm);
- return kmem_cache_alloc(PGT_CACHE(PGD_INDEX_SIZE), GFP_KERNEL);
+ return kmem_cache_alloc(PGT_CACHE(PGD_INDEX_SIZE),
+ pgtable_gfp_flags(mm, GFP_KERNEL));
}
static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
@@ -93,7 +95,8 @@ static inline void pgd_populate(struct mm_struct *mm, pgd_t *pgd, pud_t *pud)
static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr)
{
- return kmem_cache_alloc(PGT_CACHE(PUD_INDEX_SIZE), GFP_KERNEL);
+ return kmem_cache_alloc(PGT_CACHE(PUD_INDEX_SIZE),
+ pgtable_gfp_flags(mm, GFP_KERNEL));
}
static inline void pud_free(struct mm_struct *mm, pud_t *pud)
@@ -119,7 +122,8 @@ static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pud,
static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr)
{
- return kmem_cache_alloc(PGT_CACHE(PMD_CACHE_INDEX), GFP_KERNEL);
+ return kmem_cache_alloc(PGT_CACHE(PMD_CACHE_INDEX),
+ pgtable_gfp_flags(mm, GFP_KERNEL));
}
static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
@@ -168,7 +172,8 @@ static inline pgtable_t pte_alloc_one(struct mm_struct *mm,
struct page *page;
pte_t *pte;
- pte = pte_alloc_one_kernel(mm, address);
+ pte = (pte_t *)__get_free_page(pgtable_gfp_flags(mm,
+ GFP_KERNEL | __GFP_ZERO));
if (!pte)
return NULL;
page = virt_to_page(pte);
diff --git a/arch/powerpc/include/asm/nohash/64/pgalloc.h b/arch/powerpc/include/asm/nohash/64/pgalloc.h
index 897d2e1..0d5e37b 100644
--- a/arch/powerpc/include/asm/nohash/64/pgalloc.h
+++ b/arch/powerpc/include/asm/nohash/64/pgalloc.h
@@ -43,7 +43,8 @@ extern struct kmem_cache *pgtable_cache[];
static inline pgd_t *pgd_alloc(struct mm_struct *mm)
{
- return kmem_cache_alloc(PGT_CACHE(PGD_INDEX_SIZE), GFP_KERNEL);
+ return kmem_cache_alloc(PGT_CACHE(PGD_INDEX_SIZE),
+ pgtable_gfp_flags(mm, GFP_KERNEL));
}
static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
@@ -57,7 +58,8 @@ static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr)
{
- return kmem_cache_alloc(PGT_CACHE(PUD_INDEX_SIZE), GFP_KERNEL);
+ return kmem_cache_alloc(PGT_CACHE(PUD_INDEX_SIZE),
+ pgtable_gfp_flags(mm, GFP_KERNEL));
}
static inline void pud_free(struct mm_struct *mm, pud_t *pud)
@@ -96,7 +98,8 @@ static inline pgtable_t pte_alloc_one(struct mm_struct *mm,
struct page *page;
pte_t *pte;
- pte = pte_alloc_one_kernel(mm, address);
+ pte = (pte_t *)__get_free_page(
+ pgtable_gfp_flags(mm, GFP_KERNEL | __GFP_ZERO));
if (!pte)
return NULL;
page = virt_to_page(pte);
@@ -189,7 +192,8 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t table,
static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr)
{
- return kmem_cache_alloc(PGT_CACHE(PMD_CACHE_INDEX), GFP_KERNEL);
+ return kmem_cache_alloc(PGT_CACHE(PMD_CACHE_INDEX),
+ pgtable_gfp_flags(mm, GFP_KERNEL));
}
static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
diff --git a/arch/powerpc/include/asm/pgalloc.h b/arch/powerpc/include/asm/pgalloc.h
index 0413457..d795c5d 100644
--- a/arch/powerpc/include/asm/pgalloc.h
+++ b/arch/powerpc/include/asm/pgalloc.h
@@ -3,6 +3,20 @@
#include <linux/mm.h>
+#ifndef MODULE
+static inline gfp_t pgtable_gfp_flags(struct mm_struct *mm, gfp_t gfp)
+{
+ if (unlikely(mm == &init_mm))
+ return gfp;
+ return gfp | __GFP_ACCOUNT;
+}
+#else /* !MODULE */
+static inline gfp_t pgtable_gfp_flags(struct mm_struct *mm, gfp_t gfp)
+{
+ return gfp | __GFP_ACCOUNT;
+}
+#endif /* MODULE */
+
#ifdef CONFIG_PPC_BOOK3S
#include <asm/book3s/pgalloc.h>
#else
diff --git a/arch/powerpc/mm/pgtable_64.c b/arch/powerpc/mm/pgtable_64.c
index db93cf7..b322262 100644
--- a/arch/powerpc/mm/pgtable_64.c
+++ b/arch/powerpc/mm/pgtable_64.c
@@ -351,9 +351,10 @@ static pte_t *get_from_cache(struct mm_struct *mm)
static pte_t *__alloc_for_cache(struct mm_struct *mm, int kernel)
{
void *ret = NULL;
- struct page *page = alloc_page(GFP_KERNEL | __GFP_NOTRACK | __GFP_ZERO);
+ struct page *page = alloc_page(pgtable_gfp_flags(mm, PGALLOC_GFP));
if (!page)
return NULL;
+
if (!kernel && !pgtable_page_ctor(page)) {
__free_page(page);
return NULL;
--
2.9.3
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 2/3] powerpc/mm/book(e)(3s)/32: Add page table accounting
From: Balbir Singh @ 2017-05-01 6:34 UTC (permalink / raw)
To: vdavydov, mpe, oss; +Cc: linuxppc-dev, linux-mm, Balbir Singh
In-Reply-To: <20170501063438.25237-1-bsingharora@gmail.com>
Add support in pte_alloc_one() and pgd_alloc() by
passing __GFP_ACCOUNT in the flags
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
arch/powerpc/include/asm/nohash/32/pgalloc.h | 3 ++-
arch/powerpc/mm/pgtable_32.c | 2 +-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/nohash/32/pgalloc.h b/arch/powerpc/include/asm/nohash/32/pgalloc.h
index 6331392..1900d9c 100644
--- a/arch/powerpc/include/asm/nohash/32/pgalloc.h
+++ b/arch/powerpc/include/asm/nohash/32/pgalloc.h
@@ -31,7 +31,8 @@ extern struct kmem_cache *pgtable_cache[];
static inline pgd_t *pgd_alloc(struct mm_struct *mm)
{
- return kmem_cache_alloc(PGT_CACHE(PGD_INDEX_SIZE), GFP_KERNEL);
+ return kmem_cache_alloc(PGT_CACHE(PGD_INDEX_SIZE),
+ pgtable_gfp_flags(GFP_KERNEL));
}
static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
diff --git a/arch/powerpc/mm/pgtable_32.c b/arch/powerpc/mm/pgtable_32.c
index a65c0b4..dc1e0c2 100644
--- a/arch/powerpc/mm/pgtable_32.c
+++ b/arch/powerpc/mm/pgtable_32.c
@@ -60,7 +60,7 @@ pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long address)
{
struct page *ptepage;
- gfp_t flags = GFP_KERNEL | __GFP_ZERO;
+ gfp_t flags = GFP_KERNEL | __GFP_ZERO | __GFP_ACCOUNT;
ptepage = alloc_pages(flags, 0);
if (!ptepage)
--
2.9.3
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 3/3] powerpc/mm/hugetlb: Add support for page accounting
From: Balbir Singh @ 2017-05-01 6:34 UTC (permalink / raw)
To: vdavydov, mpe, oss; +Cc: linuxppc-dev, linux-mm, Balbir Singh
In-Reply-To: <20170501063438.25237-1-bsingharora@gmail.com>
Add __GFP_ACCOUNT to __hugepte_alloc()
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
---
arch/powerpc/mm/hugetlbpage.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/mm/hugetlbpage.c b/arch/powerpc/mm/hugetlbpage.c
index a4f33de..94e56b1 100644
--- a/arch/powerpc/mm/hugetlbpage.c
+++ b/arch/powerpc/mm/hugetlbpage.c
@@ -77,7 +77,7 @@ static int __hugepte_alloc(struct mm_struct *mm, hugepd_t *hpdp,
num_hugepd = 1;
}
- new = kmem_cache_zalloc(cachep, GFP_KERNEL);
+ new = kmem_cache_zalloc(cachep, pgtable_gfp_flags(mm, GFP_KERNEL));
BUG_ON(pshift > HUGEPD_SHIFT_MASK);
BUG_ON((unsigned long)new & HUGEPD_SHIFT_MASK);
--
2.9.3
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [PATCH v2] mm, zone_device: replace {get, put}_zone_device_page() with a single reference
From: Ingo Molnar @ 2017-05-01 7:12 UTC (permalink / raw)
To: Dan Williams
Cc: linux-kernel@vger.kernel.org, Linux MM, Jérôme Glisse,
Ingo Molnar, Andrew Morton, Logan Gunthorpe, Kirill Shutemov
In-Reply-To: <CAPcyv4i8WrNPzu_-Lu1uKi8NT-vj1PF0h0SW_Pi=QGn5PPhQfQ@mail.gmail.com>
* Dan Williams <dan.j.williams@intel.com> wrote:
> On Sat, Apr 29, 2017 at 7:18 AM, Ingo Molnar <mingo@kernel.org> wrote:
> >
> > * Dan Williams <dan.j.williams@intel.com> wrote:
> >
> >> Kirill points out that the calls to {get,put}_dev_pagemap() can be
> >> removed from the mm fast path if we take a single get_dev_pagemap()
> >> reference to signify that the page is alive and use the final put of the
> >> page to drop that reference.
> >>
> >> This does require some care to make sure that any waits for the
> >> percpu_ref to drop to zero occur *after* devm_memremap_page_release(),
> >> since it now maintains its own elevated reference.
> >>
> >> Cc: Ingo Molnar <mingo@redhat.com>
> >> Cc: Jerome Glisse <jglisse@redhat.com>
> >> Cc: Andrew Morton <akpm@linux-foundation.org>
> >> Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
> >> Suggested-by: Kirill Shutemov <kirill.shutemov@linux.intel.com>
> >> Tested-by: Kirill Shutemov <kirill.shutemov@linux.intel.com>
> >> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> >
> > This changelog is lacking an explanation about how this solves the crashes you
> > were seeing.
>
> Kirill? It wasn't clear to me why the conversion to generic
> get_user_pages_fast() caused the reference counts to be off.
Ok, the merge window is open and we really need this fix for x86/mm, so this is
what I've decoded:
The x86 conversion to the generic GUP code included a small change which causes
crashes and data corruption in the pmem code - not good.
The root cause is that the /dev/pmem driver code implicitly relies on the x86
get_user_pages() implementation doing a get_page() on the page refcount, because
get_page() does a get_zone_device_page() which properly refcounts pmem's separate
page struct arrays that are not present in the regular page struct structures.
(The pmem driver does this because it can cover huge memory areas.)
But the x86 conversion to the generic GUP code changed the get_page() to
page_cache_get_speculative() which is faster but doesn't do the
get_zone_device_page() call the pmem code relies on.
One way to solve the regression would be to change the generic GUP code to use
get_page(), but that would slow things down a bit and punish other generic-GUP
using architectures for an x86-ism they did not care about. (Arguably the pmem
driver was probably not working reliably for them: but nvdimm is an Intel
feature, so non-x86 exposure is probably still limited.)
So restructure the pmem code's interface with the MM instead: get rid of the
get/put_zone_device_page() distinction, integrate put_zone_device_page() into
__put_page() and and restructure the pmem completion-wait and teardown machinery.
This speeds up things while also making the pmem refcounting more robust going
forward.
... is this extension to the changelog correct?
I'll apply this for the time being - but can still amend the text before sending
it to Linus later today.
Thanks,
Ingo
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 2/3] mm/slub: wrap cpu_slab->partial in CONFIG_SLUB_CPU_PARTIAL
From: Wei Yang @ 2017-05-01 7:39 UTC (permalink / raw)
To: Matthew Wilcox
Cc: Wei Yang, cl, penberg, rientjes, iamjoonsoo.kim, akpm, linux-mm,
linux-kernel
In-Reply-To: <20170501024103.GI27790@bombadil.infradead.org>
[-- Attachment #1: Type: text/plain, Size: 2151 bytes --]
On Sun, Apr 30, 2017 at 07:41:03PM -0700, Matthew Wilcox wrote:
>On Sun, Apr 30, 2017 at 07:31:51PM +0800, Wei Yang wrote:
>> @@ -2302,7 +2302,11 @@ static bool has_cpu_slab(int cpu, void *info)
>> struct kmem_cache *s = info;
>> struct kmem_cache_cpu *c = per_cpu_ptr(s->cpu_slab, cpu);
>>
>> - return c->page || c->partial;
>> + return c->page
>> +#ifdef CONFIG_SLUB_CPU_PARTIAL
>> + || c->partial
>> +#endif
>> + ;
>> }
>
>No. No way. This is disgusting.
>
Thanks for your comment. I believe you are right.
>The right way to do this is to create an accessor like this:
>
>#ifdef CONFIG_SLUB_CPU_PARTIAL
>#define slub_cpu_partial(c) ((c)->partial)
>#else
>#define slub_cpu_partial(c) 0
Since partial is a pointer to a page, would this be more proper?
#define slub_cpu_partial(c) NULL
>#endif
>
>And then the above becomes:
>
>- return c->page || c->partial;
>+ return c->page || slub_cpu_partial(c);
>
>All the other ifdefs go away, apart from these two:
Looks most of the ifdefs could be replaced by this format, while not all of
them. For example, the sysfs entry.
I would form another version with your suggestion.
Welcome any other comments :-)
>
>> @@ -4980,6 +4990,7 @@ static ssize_t objects_partial_show(struct kmem_cache *s, char *buf)
>> }
>> SLAB_ATTR_RO(objects_partial);
>>
>> +#ifdef CONFIG_SLUB_CPU_PARTIAL
>> static ssize_t slabs_cpu_partial_show(struct kmem_cache *s, char *buf)
>> {
>> int objects = 0;
>> @@ -5010,6 +5021,7 @@ static ssize_t slabs_cpu_partial_show(struct kmem_cache *s, char *buf)
>> return len + sprintf(buf + len, "\n");
>> }
>> SLAB_ATTR_RO(slabs_cpu_partial);
>> +#endif
>>
>> static ssize_t reclaim_account_show(struct kmem_cache *s, char *buf)
>> {
>> @@ -5364,7 +5376,9 @@ static struct attribute *slab_attrs[] = {
>> &destroy_by_rcu_attr.attr,
>> &shrink_attr.attr,
>> &reserved_attr.attr,
>> +#ifdef CONFIG_SLUB_CPU_PARTIAL
>> &slabs_cpu_partial_attr.attr,
>> +#endif
>> #ifdef CONFIG_SLUB_DEBUG
>> &total_objects_attr.attr,
>> &slabs_attr.attr,
--
Wei Yang
Help you, Help me
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH 2/3] mm/slub: wrap cpu_slab->partial in CONFIG_SLUB_CPU_PARTIAL
From: Wei Yang @ 2017-05-01 8:20 UTC (permalink / raw)
To: Matthew Wilcox
Cc: Wei Yang, cl, penberg, rientjes, iamjoonsoo.kim, akpm, linux-mm,
linux-kernel
In-Reply-To: <20170501024103.GI27790@bombadil.infradead.org>
[-- Attachment #1: Type: text/plain, Size: 2072 bytes --]
On Sun, Apr 30, 2017 at 07:41:03PM -0700, Matthew Wilcox wrote:
>On Sun, Apr 30, 2017 at 07:31:51PM +0800, Wei Yang wrote:
>> @@ -2302,7 +2302,11 @@ static bool has_cpu_slab(int cpu, void *info)
>> struct kmem_cache *s = info;
>> struct kmem_cache_cpu *c = per_cpu_ptr(s->cpu_slab, cpu);
>>
>> - return c->page || c->partial;
>> + return c->page
>> +#ifdef CONFIG_SLUB_CPU_PARTIAL
>> + || c->partial
>> +#endif
>> + ;
>> }
>
>No. No way. This is disgusting.
>
>The right way to do this is to create an accessor like this:
>
>#ifdef CONFIG_SLUB_CPU_PARTIAL
>#define slub_cpu_partial(c) ((c)->partial)
>#else
>#define slub_cpu_partial(c) 0
>#endif
>
>And then the above becomes:
>
>- return c->page || c->partial;
>+ return c->page || slub_cpu_partial(c);
>
>All the other ifdefs go away, apart from these two:
>
Matthew
I have tried to replace the code with slub_cpu_partial(), it works fine on
most of cases except two:
1. slub_cpu_partial(c) = page->next;
2. page = READ_ONCE(slub_cpu_partial(c));
The sysfs part works fine.
So if you agree, I would leave these two parts as v1.
>> @@ -4980,6 +4990,7 @@ static ssize_t objects_partial_show(struct kmem_cache *s, char *buf)
>> }
>> SLAB_ATTR_RO(objects_partial);
>>
>> +#ifdef CONFIG_SLUB_CPU_PARTIAL
>> static ssize_t slabs_cpu_partial_show(struct kmem_cache *s, char *buf)
>> {
>> int objects = 0;
>> @@ -5010,6 +5021,7 @@ static ssize_t slabs_cpu_partial_show(struct kmem_cache *s, char *buf)
>> return len + sprintf(buf + len, "\n");
>> }
>> SLAB_ATTR_RO(slabs_cpu_partial);
>> +#endif
>>
>> static ssize_t reclaim_account_show(struct kmem_cache *s, char *buf)
>> {
>> @@ -5364,7 +5376,9 @@ static struct attribute *slab_attrs[] = {
>> &destroy_by_rcu_attr.attr,
>> &shrink_attr.attr,
>> &reserved_attr.attr,
>> +#ifdef CONFIG_SLUB_CPU_PARTIAL
>> &slabs_cpu_partial_attr.attr,
>> +#endif
>> #ifdef CONFIG_SLUB_DEBUG
>> &total_objects_attr.attr,
>> &slabs_attr.attr,
--
Wei Yang
Help you, Help me
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH v2] mm, zone_device: replace {get, put}_zone_device_page() with a single reference
From: Kirill A. Shutemov @ 2017-05-01 9:33 UTC (permalink / raw)
To: Ingo Molnar
Cc: Dan Williams, linux-kernel@vger.kernel.org, Linux MM,
Jérôme Glisse, Ingo Molnar, Andrew Morton,
Logan Gunthorpe, Kirill Shutemov
In-Reply-To: <20170501071259.5vya524wcdddm42b@gmail.com>
On Mon, May 01, 2017 at 09:12:59AM +0200, Ingo Molnar wrote:
> ... is this extension to the changelog correct?
Looks good to me.
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2 2/3] powerpc/mm/book(e)(3s)/32: Add page table accounting
From: kbuild test robot @ 2017-05-01 10:02 UTC (permalink / raw)
To: Balbir Singh; +Cc: kbuild-all, vdavydov, mpe, oss, linuxppc-dev, linux-mm
In-Reply-To: <20170501063438.25237-3-bsingharora@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 3054 bytes --]
Hi Balbir,
[auto build test ERROR on powerpc/next]
[also build test ERROR on v4.11 next-20170428]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Balbir-Singh/powerpc-mm-book-e-3s-64-Add-page-table-accounting/20170501-143900
base: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
config: powerpc-virtex5_defconfig (attached as .config)
compiler: powerpc-linux-gnu-gcc (Debian 6.1.1-9) 6.1.1 20160705
reproduce:
wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=powerpc
All error/warnings (new ones prefixed by >>):
In file included from include/linux/mm.h:9:0,
from include/linux/cpuset.h:16,
from kernel//sched/core.c:13:
arch/powerpc/include/asm/nohash/32/pgalloc.h: In function 'pgd_alloc':
>> include/linux/gfp.h:240:20: warning: passing argument 1 of 'pgtable_gfp_flags' makes pointer from integer without a cast [-Wint-conversion]
#define GFP_KERNEL (__GFP_RECLAIM | __GFP_IO | __GFP_FS)
^
>> arch/powerpc/include/asm/nohash/32/pgalloc.h:35:22: note: in expansion of macro 'GFP_KERNEL'
pgtable_gfp_flags(GFP_KERNEL));
^~~~~~~~~~
In file included from arch/powerpc/include/asm/tlb.h:19:0,
from kernel//sched/core.c:30:
arch/powerpc/include/asm/pgalloc.h:7:21: note: expected 'struct mm_struct *' but argument is of type 'unsigned int'
static inline gfp_t pgtable_gfp_flags(struct mm_struct *mm, gfp_t gfp)
^~~~~~~~~~~~~~~~~
In file included from arch/powerpc/include/asm/nohash/pgalloc.h:21:0,
from arch/powerpc/include/asm/pgalloc.h:23,
from arch/powerpc/include/asm/tlb.h:19,
from kernel//sched/core.c:30:
>> arch/powerpc/include/asm/nohash/32/pgalloc.h:35:4: error: too few arguments to function 'pgtable_gfp_flags'
pgtable_gfp_flags(GFP_KERNEL));
^~~~~~~~~~~~~~~~~
In file included from arch/powerpc/include/asm/tlb.h:19:0,
from kernel//sched/core.c:30:
arch/powerpc/include/asm/pgalloc.h:7:21: note: declared here
static inline gfp_t pgtable_gfp_flags(struct mm_struct *mm, gfp_t gfp)
^~~~~~~~~~~~~~~~~
vim +/pgtable_gfp_flags +35 arch/powerpc/include/asm/nohash/32/pgalloc.h
29 pgtable_cache[(shift) - 1]; \
30 })
31
32 static inline pgd_t *pgd_alloc(struct mm_struct *mm)
33 {
34 return kmem_cache_alloc(PGT_CACHE(PGD_INDEX_SIZE),
> 35 pgtable_gfp_flags(GFP_KERNEL));
36 }
37
38 static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 13899 bytes --]
^ permalink raw reply
* Re: [PATCH v2 2/3] powerpc/mm/book(e)(3s)/32: Add page table accounting
From: kbuild test robot @ 2017-05-01 10:31 UTC (permalink / raw)
To: Balbir Singh; +Cc: kbuild-all, vdavydov, mpe, oss, linuxppc-dev, linux-mm
In-Reply-To: <20170501063438.25237-3-bsingharora@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 5531 bytes --]
Hi Balbir,
[auto build test ERROR on powerpc/next]
[also build test ERROR on v4.11 next-20170428]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Balbir-Singh/powerpc-mm-book-e-3s-64-Add-page-table-accounting/20170501-143900
base: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
config: powerpc-virtex5_defconfig (attached as .config)
compiler: powerpc-linux-gnu-gcc (Debian 6.1.1-9) 6.1.1 20160705
reproduce:
wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=powerpc
All error/warnings (new ones prefixed by >>):
In file included from arch/powerpc/mm/mem.c:25:0:
arch/powerpc/include/asm/nohash/32/pgalloc.h: In function 'pgd_alloc':
>> include/linux/gfp.h:240:20: error: passing argument 1 of 'pgtable_gfp_flags' makes pointer from integer without a cast [-Werror=int-conversion]
#define GFP_KERNEL (__GFP_RECLAIM | __GFP_IO | __GFP_FS)
^
arch/powerpc/include/asm/nohash/32/pgalloc.h:35:22: note: in expansion of macro 'GFP_KERNEL'
pgtable_gfp_flags(GFP_KERNEL));
^~~~~~~~~~
In file included from arch/powerpc/mm/mem.c:40:0:
arch/powerpc/include/asm/pgalloc.h:7:21: note: expected 'struct mm_struct *' but argument is of type 'unsigned int'
static inline gfp_t pgtable_gfp_flags(struct mm_struct *mm, gfp_t gfp)
^~~~~~~~~~~~~~~~~
In file included from arch/powerpc/include/asm/nohash/pgalloc.h:21:0,
from arch/powerpc/include/asm/pgalloc.h:23,
from arch/powerpc/mm/mem.c:40:
arch/powerpc/include/asm/nohash/32/pgalloc.h:35:4: error: too few arguments to function 'pgtable_gfp_flags'
pgtable_gfp_flags(GFP_KERNEL));
^~~~~~~~~~~~~~~~~
In file included from arch/powerpc/mm/mem.c:40:0:
arch/powerpc/include/asm/pgalloc.h:7:21: note: declared here
static inline gfp_t pgtable_gfp_flags(struct mm_struct *mm, gfp_t gfp)
^~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
--
In file included from include/linux/slab.h:14:0,
from kernel/fork.c:14:
arch/powerpc/include/asm/nohash/32/pgalloc.h: In function 'pgd_alloc':
include/linux/gfp.h:240:20: warning: passing argument 1 of 'pgtable_gfp_flags' makes pointer from integer without a cast [-Wint-conversion]
#define GFP_KERNEL (__GFP_RECLAIM | __GFP_IO | __GFP_FS)
^
arch/powerpc/include/asm/nohash/32/pgalloc.h:35:22: note: in expansion of macro 'GFP_KERNEL'
pgtable_gfp_flags(GFP_KERNEL));
^~~~~~~~~~
In file included from kernel/fork.c:92:0:
arch/powerpc/include/asm/pgalloc.h:7:21: note: expected 'struct mm_struct *' but argument is of type 'unsigned int'
static inline gfp_t pgtable_gfp_flags(struct mm_struct *mm, gfp_t gfp)
^~~~~~~~~~~~~~~~~
In file included from arch/powerpc/include/asm/nohash/pgalloc.h:21:0,
from arch/powerpc/include/asm/pgalloc.h:23,
from kernel/fork.c:92:
arch/powerpc/include/asm/nohash/32/pgalloc.h:35:4: error: too few arguments to function 'pgtable_gfp_flags'
pgtable_gfp_flags(GFP_KERNEL));
^~~~~~~~~~~~~~~~~
In file included from kernel/fork.c:92:0:
arch/powerpc/include/asm/pgalloc.h:7:21: note: declared here
static inline gfp_t pgtable_gfp_flags(struct mm_struct *mm, gfp_t gfp)
^~~~~~~~~~~~~~~~~
In file included from arch/powerpc/include/asm/nohash/pgalloc.h:21:0,
from arch/powerpc/include/asm/pgalloc.h:23,
from kernel/fork.c:92:
>> arch/powerpc/include/asm/nohash/32/pgalloc.h:36:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
vim +/pgtable_gfp_flags +240 include/linux/gfp.h
251603549 Vlastimil Babka 2016-07-28 234 * compound allocations that will generally fail quickly if memory is not
251603549 Vlastimil Babka 2016-07-28 235 * available and will not wake kswapd/kcompactd on failure. The _LIGHT
251603549 Vlastimil Babka 2016-07-28 236 * version does not attempt reclaim/compaction at all and is by default used
251603549 Vlastimil Babka 2016-07-28 237 * in page fault path, while the non-light is used by khugepaged.
d0164adc8 Mel Gorman 2015-11-06 238 */
d0164adc8 Mel Gorman 2015-11-06 239 #define GFP_ATOMIC (__GFP_HIGH|__GFP_ATOMIC|__GFP_KSWAPD_RECLAIM)
dd56b0464 Mel Gorman 2015-11-06 @240 #define GFP_KERNEL (__GFP_RECLAIM | __GFP_IO | __GFP_FS)
a9bb7e620 Vladimir Davydov 2016-01-14 241 #define GFP_KERNEL_ACCOUNT (GFP_KERNEL | __GFP_ACCOUNT)
d0164adc8 Mel Gorman 2015-11-06 242 #define GFP_NOWAIT (__GFP_KSWAPD_RECLAIM)
71baba4b9 Mel Gorman 2015-11-06 243 #define GFP_NOIO (__GFP_RECLAIM)
:::::: The code at line 240 was first introduced by commit
:::::: dd56b046426760aa0c852ad6e4b6b07891222d65 mm: page_alloc: hide some GFP internals and document the bits and flag combinations
:::::: TO: Mel Gorman <mgorman@techsingularity.net>
:::::: CC: Linus Torvalds <torvalds@linux-foundation.org>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 13899 bytes --]
^ permalink raw reply
* Re: [PATCH -mm -v10 1/3] mm, THP, swap: Delay splitting THP during swap out
From: Johannes Weiner @ 2017-05-01 10:44 UTC (permalink / raw)
To: Minchan Kim
Cc: Huang, Ying, Andrew Morton, linux-mm, linux-kernel,
Andrea Arcangeli, Ebru Akagunduz, Michal Hocko, Tejun Heo,
Hugh Dickins, Shaohua Li, Rik van Riel, cgroups
In-Reply-To: <20170428084044.GB19510@bbox>
On Fri, Apr 28, 2017 at 05:40:44PM +0900, Minchan Kim wrote:
> However, get_swap_page is ugly now. The caller should take care of
> failure and should retry after split. I hope get_swap_page includes
> split and retry logic in itself without reling on the caller.
I think this makes the interface terrible. It's an allocation function
to which you pass a reference object for size - and if the allocation
doesn't succeed it'll split your reference object to make it fit?
That's a nasty side effect for this function to have.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2] mm, zone_device: replace {get, put}_zone_device_page() with a single reference
From: Kirill A. Shutemov @ 2017-05-01 10:23 UTC (permalink / raw)
To: Jerome Glisse
Cc: Dan Williams, Ingo Molnar, linux-kernel@vger.kernel.org, Linux MM,
Ingo Molnar, Andrew Morton, Logan Gunthorpe, Kirill Shutemov
In-Reply-To: <20170430231421.GA15163@redhat.com>
On Sun, Apr 30, 2017 at 07:14:24PM -0400, Jerome Glisse wrote:
> On Sat, Apr 29, 2017 at 01:17:26PM +0300, Kirill A. Shutemov wrote:
> > On Fri, Apr 28, 2017 at 03:33:07PM -0400, Jerome Glisse wrote:
> > > On Fri, Apr 28, 2017 at 12:22:24PM -0700, Dan Williams wrote:
> > > > Are you sure about needing to hook the 2 -> 1 transition? Could we
> > > > change ZONE_DEVICE pages to not have an elevated reference count when
> > > > they are created so you can keep the HMM references out of the mm hot
> > > > path?
> > >
> > > 100% sure on that :) I need to callback into driver for 2->1 transition
> > > no way around that. If we change ZONE_DEVICE to not have an elevated
> > > reference count that you need to make a lot more change to mm so that
> > > ZONE_DEVICE is never use as fallback for memory allocation. Also need
> > > to make change to be sure that ZONE_DEVICE page never endup in one of
> > > the path that try to put them back on lru. There is a lot of place that
> > > would need to be updated and it would be highly intrusive and add a
> > > lot of special cases to other hot code path.
> >
> > Could you explain more on where the requirement comes from or point me to
> > where I can read about this.
> >
>
> HMM ZONE_DEVICE pages are use like other pages (anonymous or file back page)
> in _any_ vma. So i need to know when a page is freed ie either as result of
> unmap, exit or migration or anything that would free the memory. For zone
> device a page is free once its refcount reach 1 so i need to catch refcount
> transition from 2->1
What if we would rework zone device to have pages with refcount 0 at
start?
> This is the only way i can inform the device that the page is now free. See
>
> https://cgit.freedesktop.org/~glisse/linux/commit/?h=hmm-v21&id=52da8fe1a088b87b5321319add79e43b8372ed7d
>
> There is _no_ way around that.
I'm still not convinced that it's impossible.
Could you describe lifecycle for pages in case of HMM?
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2 2/3] powerpc/mm/book(e)(3s)/32: Add page table accounting
From: Balbir Singh @ 2017-05-01 11:09 UTC (permalink / raw)
To: kbuild test robot
Cc: kbuild-all, Vladimir Davydov, Michael Ellerman, Scott Wood,
open list:LINUX FOR POWERPC (32-BIT AND 64-BIT), linux-mm
In-Reply-To: <201705011829.pgKWNzqt%fengguang.wu@intel.com>
On Mon, May 1, 2017 at 8:31 PM, kbuild test robot <lkp@intel.com> wrote:
> Hi Balbir,
>
> [auto build test ERROR on powerpc/next]
> [also build test ERROR on v4.11 next-20170428]
> [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
>
> url: https://github.com/0day-ci/linux/commits/Balbir-Singh/powerpc-mm-book-e-3s-64-Add-page-table-accounting/20170501-143900
> base: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
> config: powerpc-virtex5_defconfig (attached as .config)
> compiler: powerpc-linux-gnu-gcc (Debian 6.1.1-9) 6.1.1 20160705
> reproduce:
> wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
> chmod +x ~/bin/make.cross
> # save the attached .config to linux build tree
> make.cross ARCH=powerpc
>
> All error/warnings (new ones prefixed by >>):
>
> In file included from arch/powerpc/mm/mem.c:25:0:
> arch/powerpc/include/asm/nohash/32/pgalloc.h: In function 'pgd_alloc':
>>> include/linux/gfp.h:240:20: error: passing argument 1 of 'pgtable_gfp_flags' makes pointer from integer without a cast [-Werror=int-conversion]
> #define GFP_KERNEL (__GFP_RECLAIM | __GFP_IO | __GFP_FS)
> ^
> arch/powerpc/include/asm/nohash/32/pgalloc.h:35:22: note: in expansion of macro 'GFP_KERNEL'
> pgtable_gfp_flags(GFP_KERNEL));
That's a silly build error that escaped my build scripts, I'll send
out a better v3 with 32 bits fixed.
Balbir
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2] mm, zone_device: replace {get, put}_zone_device_page() with a single reference
From: Jerome Glisse @ 2017-05-01 13:55 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Dan Williams, Ingo Molnar, linux-kernel@vger.kernel.org, Linux MM,
Ingo Molnar, Andrew Morton, Logan Gunthorpe, Kirill Shutemov
In-Reply-To: <20170501102359.abopw7hpd4eb6x2w@node.shutemov.name>
On Mon, May 01, 2017 at 01:23:59PM +0300, Kirill A. Shutemov wrote:
> On Sun, Apr 30, 2017 at 07:14:24PM -0400, Jerome Glisse wrote:
> > On Sat, Apr 29, 2017 at 01:17:26PM +0300, Kirill A. Shutemov wrote:
> > > On Fri, Apr 28, 2017 at 03:33:07PM -0400, Jerome Glisse wrote:
> > > > On Fri, Apr 28, 2017 at 12:22:24PM -0700, Dan Williams wrote:
> > > > > Are you sure about needing to hook the 2 -> 1 transition? Could we
> > > > > change ZONE_DEVICE pages to not have an elevated reference count when
> > > > > they are created so you can keep the HMM references out of the mm hot
> > > > > path?
> > > >
> > > > 100% sure on that :) I need to callback into driver for 2->1 transition
> > > > no way around that. If we change ZONE_DEVICE to not have an elevated
> > > > reference count that you need to make a lot more change to mm so that
> > > > ZONE_DEVICE is never use as fallback for memory allocation. Also need
> > > > to make change to be sure that ZONE_DEVICE page never endup in one of
> > > > the path that try to put them back on lru. There is a lot of place that
> > > > would need to be updated and it would be highly intrusive and add a
> > > > lot of special cases to other hot code path.
> > >
> > > Could you explain more on where the requirement comes from or point me to
> > > where I can read about this.
> > >
> >
> > HMM ZONE_DEVICE pages are use like other pages (anonymous or file back page)
> > in _any_ vma. So i need to know when a page is freed ie either as result of
> > unmap, exit or migration or anything that would free the memory. For zone
> > device a page is free once its refcount reach 1 so i need to catch refcount
> > transition from 2->1
>
> What if we would rework zone device to have pages with refcount 0 at
> start?
That is a _lot_ of work from top of my head because it would need changes
to a lot of places and likely more hot code path that simply adding some-
thing to put_page() note that i only need something in put_page() i do not
need anything in the get page path. Is adding a conditional branch for
HMM pages in put_page() that much of a problem ?
> > This is the only way i can inform the device that the page is now free. See
> >
> > https://cgit.freedesktop.org/~glisse/linux/commit/?h=hmm-v21&id=52da8fe1a088b87b5321319add79e43b8372ed7d
> >
> > There is _no_ way around that.
>
> I'm still not convinced that it's impossible.
>
> Could you describe lifecycle for pages in case of HMM?
Process malloc something, end it over to some function in the program
that use the GPU that function call GPU API (OpenCL, CUDA, ...) that
trigger a migration to device memory.
So in the kernel you get a migration like any existing migration,
original page is unmap, if refcount is all ok (no pin) then a device
page is allocated and thing are migrated to device memory.
What happen after is unknown. Either userspace/kernel driver decide
to migrate back to system memory, either there is an munmap, either
there is a CPU page fault, ... So from that point on the device page
as the exact same life as a regular page.
Above i describe the migrate case, but you can also have new memory
allocation that directly allocate device memory. For instance if the
GPU do a page fault on an address that isn't back by anything then
we can directly allocate a device page. No migration involve in that
case.
HMM pages are like any other pages in most respect. Exception are:
- no GUP
- no KSM
- no lru reclaim
- no NUMA balancing
- no regular migration (existing migrate_page)
The fact that minimum refcount for ZONE_DEVICE is 1 already gives
us for free most of the above exception. To convert the refcount to
be like other pages would mean that all of the above would need to
be audited and probably modify to ignore ZONE_DEVICE pages (i am
pretty sure Dan do not want any of the above either).
Cheers,
Jerome
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 2/3] mm/slub: wrap cpu_slab->partial in CONFIG_SLUB_CPU_PARTIAL
From: Matthew Wilcox @ 2017-05-01 14:39 UTC (permalink / raw)
To: Wei Yang
Cc: cl, penberg, rientjes, iamjoonsoo.kim, akpm, linux-mm,
linux-kernel
In-Reply-To: <20170501082005.GA2006@WeideMacBook-Pro.local>
On Mon, May 01, 2017 at 04:20:05PM +0800, Wei Yang wrote:
> I have tried to replace the code with slub_cpu_partial(), it works fine on
> most of cases except two:
>
> 1. slub_cpu_partial(c) = page->next;
New accessor: slub_set_cpu_partial(c, p)
> 2. page = READ_ONCE(slub_cpu_partial(c));
OK, that one I haven't seen an existing pattern for yet.
slub_cpu_partial_read_once(c)?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 2/3] mm/slub: wrap cpu_slab->partial in CONFIG_SLUB_CPU_PARTIAL
From: Wei Yang @ 2017-05-01 15:15 UTC (permalink / raw)
To: Matthew Wilcox
Cc: Wei Yang, cl, penberg, rientjes, iamjoonsoo.kim, akpm, linux-mm,
linux-kernel
In-Reply-To: <20170501143930.GJ27790@bombadil.infradead.org>
[-- Attachment #1: Type: text/plain, Size: 542 bytes --]
On Mon, May 01, 2017 at 07:39:30AM -0700, Matthew Wilcox wrote:
>On Mon, May 01, 2017 at 04:20:05PM +0800, Wei Yang wrote:
>> I have tried to replace the code with slub_cpu_partial(), it works fine on
>> most of cases except two:
>>
>> 1. slub_cpu_partial(c) = page->next;
>
>New accessor: slub_set_cpu_partial(c, p)
>
>> 2. page = READ_ONCE(slub_cpu_partial(c));
>
>OK, that one I haven't seen an existing pattern for yet.
>slub_cpu_partial_read_once(c)?
Thanks~ You are really a genius.
--
Wei Yang
Help you, Help me
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH 3/3] mm/slub: wrap kmem_cache->cpu_partial in config CONFIG_SLUB_CPU_PARTIAL
From: Wei Yang @ 2017-05-01 15:37 UTC (permalink / raw)
Cc: cl, penberg, rientjes, iamjoonsoo.kim, akpm, linux-mm,
linux-kernel, Wei Yang
In-Reply-To: <20170430113152.6590-4-richard.weiyang@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 5871 bytes --]
On Sun, Apr 30, 2017 at 07:31:52PM +0800, Wei Yang wrote:
>kmem_cache->cpu_partial is just used when CONFIG_SLUB_CPU_PARTIAL is set,
>so wrap it with config CONFIG_SLUB_CPU_PARTIAL will save some space
>on 32bit arch.
>
>This patch wrap kmem_cache->cpu_partial in config CONFIG_SLUB_CPU_PARTIAL
>and wrap its sysfs too.
>
>Signed-off-by: Wei Yang <richard.weiyang@gmail.com>
>---
> include/linux/slub_def.h | 2 ++
> mm/slub.c | 72 +++++++++++++++++++++++++++++-------------------
> 2 files changed, 46 insertions(+), 28 deletions(-)
>
>diff --git a/include/linux/slub_def.h b/include/linux/slub_def.h
>index 0debd8df1a7d..477ab99800ed 100644
>--- a/include/linux/slub_def.h
>+++ b/include/linux/slub_def.h
>@@ -69,7 +69,9 @@ struct kmem_cache {
> int size; /* The size of an object including meta data */
> int object_size; /* The size of an object without meta data */
> int offset; /* Free pointer offset. */
>+#ifdef CONFIG_SLUB_CPU_PARTIAL
> int cpu_partial; /* Number of per cpu partial objects to keep around */
>+#endif
> struct kmem_cache_order_objects oo;
>
> /* Allocation and freeing of slabs */
>diff --git a/mm/slub.c b/mm/slub.c
>index fde499b6dad8..94978f27882a 100644
>--- a/mm/slub.c
>+++ b/mm/slub.c
>@@ -1829,7 +1829,10 @@ static void *get_partial_node(struct kmem_cache *s, struct kmem_cache_node *n,
> stat(s, CPU_PARTIAL_NODE);
> }
> if (!kmem_cache_has_cpu_partial(s)
>- || available > s->cpu_partial / 2)
>+#ifdef CONFIG_SLUB_CPU_PARTIAL
>+ || available > s->cpu_partial / 2
>+#endif
>+ )
> break;
Matthew,
I plan to change this one with the same idea you mentioned in previous reply.
While one special "technique" is how to name it.
How about name this one
slub_cpu_partial()
And rename the previous one
slub_percpu_partial()
This is really hard to say which one is better :-(
Not sure whether you have some insight in this.
>
> }
>@@ -3418,6 +3421,39 @@ static void set_min_partial(struct kmem_cache *s, unsigned long min)
> s->min_partial = min;
> }
>
>+static void set_cpu_partial(struct kmem_cache *s)
>+{
>+#ifdef CONFIG_SLUB_CPU_PARTIAL
>+ /*
>+ * cpu_partial determined the maximum number of objects kept in the
>+ * per cpu partial lists of a processor.
>+ *
>+ * Per cpu partial lists mainly contain slabs that just have one
>+ * object freed. If they are used for allocation then they can be
>+ * filled up again with minimal effort. The slab will never hit the
>+ * per node partial lists and therefore no locking will be required.
>+ *
>+ * This setting also determines
>+ *
>+ * A) The number of objects from per cpu partial slabs dumped to the
>+ * per node list when we reach the limit.
>+ * B) The number of objects in cpu partial slabs to extract from the
>+ * per node list when we run out of per cpu objects. We only fetch
>+ * 50% to keep some capacity around for frees.
>+ */
>+ if (!kmem_cache_has_cpu_partial(s))
>+ s->cpu_partial = 0;
>+ else if (s->size >= PAGE_SIZE)
>+ s->cpu_partial = 2;
>+ else if (s->size >= 1024)
>+ s->cpu_partial = 6;
>+ else if (s->size >= 256)
>+ s->cpu_partial = 13;
>+ else
>+ s->cpu_partial = 30;
>+#endif
>+}
>+
> /*
> * calculate_sizes() determines the order and the distribution of data within
> * a slab object.
>@@ -3576,33 +3612,7 @@ static int kmem_cache_open(struct kmem_cache *s, unsigned long flags)
> */
> set_min_partial(s, ilog2(s->size) / 2);
>
>- /*
>- * cpu_partial determined the maximum number of objects kept in the
>- * per cpu partial lists of a processor.
>- *
>- * Per cpu partial lists mainly contain slabs that just have one
>- * object freed. If they are used for allocation then they can be
>- * filled up again with minimal effort. The slab will never hit the
>- * per node partial lists and therefore no locking will be required.
>- *
>- * This setting also determines
>- *
>- * A) The number of objects from per cpu partial slabs dumped to the
>- * per node list when we reach the limit.
>- * B) The number of objects in cpu partial slabs to extract from the
>- * per node list when we run out of per cpu objects. We only fetch
>- * 50% to keep some capacity around for frees.
>- */
>- if (!kmem_cache_has_cpu_partial(s))
>- s->cpu_partial = 0;
>- else if (s->size >= PAGE_SIZE)
>- s->cpu_partial = 2;
>- else if (s->size >= 1024)
>- s->cpu_partial = 6;
>- else if (s->size >= 256)
>- s->cpu_partial = 13;
>- else
>- s->cpu_partial = 30;
>+ set_cpu_partial(s);
>
> #ifdef CONFIG_NUMA
> s->remote_node_defrag_ratio = 1000;
>@@ -3989,7 +3999,9 @@ void __kmemcg_cache_deactivate(struct kmem_cache *s)
> * Disable empty slabs caching. Used to avoid pinning offline
> * memory cgroups by kmem pages that can be freed.
> */
>+#ifdef CONFIG_SLUB_CPU_PARTIAL
> s->cpu_partial = 0;
>+#endif
> s->min_partial = 0;
>
> /*
>@@ -4929,6 +4941,7 @@ static ssize_t min_partial_store(struct kmem_cache *s, const char *buf,
> }
> SLAB_ATTR(min_partial);
>
>+#ifdef CONFIG_SLUB_CPU_PARTIAL
> static ssize_t cpu_partial_show(struct kmem_cache *s, char *buf)
> {
> return sprintf(buf, "%u\n", s->cpu_partial);
>@@ -4951,6 +4964,7 @@ static ssize_t cpu_partial_store(struct kmem_cache *s, const char *buf,
> return length;
> }
> SLAB_ATTR(cpu_partial);
>+#endif
>
> static ssize_t ctor_show(struct kmem_cache *s, char *buf)
> {
>@@ -5363,7 +5377,9 @@ static struct attribute *slab_attrs[] = {
> &objs_per_slab_attr.attr,
> &order_attr.attr,
> &min_partial_attr.attr,
>+#ifdef CONFIG_SLUB_CPU_PARTIAL
> &cpu_partial_attr.attr,
>+#endif
> &objects_attr.attr,
> &objects_partial_attr.attr,
> &partial_attr.attr,
>--
>2.11.0
--
Wei Yang
Help you, Help me
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH 1/2] dax: prevent invalidation of mapped DAX entries
From: Ross Zwisler @ 2017-05-01 16:54 UTC (permalink / raw)
To: Jan Kara
Cc: Ross Zwisler, Andrew Morton, linux-kernel, Alexander Viro,
Alexey Kuznetsov, Andrey Ryabinin, Anna Schumaker,
Christoph Hellwig, Dan Williams, Darrick J. Wong,
Eric Van Hensbergen, Jens Axboe, Johannes Weiner,
Konrad Rzeszutek Wilk, Latchesar Ionkov, linux-cifs,
linux-fsdevel, linux-mm, linux-nfs, linux-nvdimm, Matthew Wilcox,
Ron Minnich, samba-technical, Steve French, Trond Myklebust,
v9fs-developer
In-Reply-To: <20170425101041.GG2793@quack2.suse.cz>
On Tue, Apr 25, 2017 at 12:10:41PM +0200, Jan Kara wrote:
> On Thu 20-04-17 21:44:36, Ross Zwisler wrote:
> > dax_invalidate_mapping_entry() currently removes DAX exceptional entries
> > only if they are clean and unlocked. This is done via:
> >
> > invalidate_mapping_pages()
> > invalidate_exceptional_entry()
> > dax_invalidate_mapping_entry()
> >
> > However, for page cache pages removed in invalidate_mapping_pages() there
> > is an additional criteria which is that the page must not be mapped. This
> > is noted in the comments above invalidate_mapping_pages() and is checked in
> > invalidate_inode_page().
> >
> > For DAX entries this means that we can can end up in a situation where a
> > DAX exceptional entry, either a huge zero page or a regular DAX entry,
> > could end up mapped but without an associated radix tree entry. This is
> > inconsistent with the rest of the DAX code and with what happens in the
> > page cache case.
> >
> > We aren't able to unmap the DAX exceptional entry because according to its
> > comments invalidate_mapping_pages() isn't allowed to block, and
> > unmap_mapping_range() takes a write lock on the mapping->i_mmap_rwsem.
> >
> > Since we essentially never have unmapped DAX entries to evict from the
> > radix tree, just remove dax_invalidate_mapping_entry().
> >
> > Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
> > Fixes: c6dcf52c23d2 ("mm: Invalidate DAX radix tree entries only if appropriate")
> > Reported-by: Jan Kara <jack@suse.cz>
> > Cc: <stable@vger.kernel.org> [4.10+]
>
> Just as a side note - we wouldn't really have to unmap the mapping range
> covered by the DAX exceptional entry. It would be enough to find out
> whether such range is mapped and bail out in that case. But that would
> still be pretty expensive for DAX - we'd have to do rmap walk similar as in
> dax_mapping_entry_mkclean() and IMHO it is not worth it. So I agree with
> what you did. You can add:
>
> Reviewed-by: Jan Kara <jack@suse.cz>
Yep, that makes sense. Thanks for the review.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox