* [RFC PATCH 1/2] drm: add support for for clk and de polarity
From: Manfred Schlaegl @ 2015-07-15 15:50 UTC (permalink / raw)
To: David Airlie, Philipp Zabel, Jean-Christophe Plagniol-Villard,
Tomi Valkeinen
Cc: dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA, Manfred Schlaegl,
Steve Longerbeam, Deepak Das, Jiada Wang,
linux-fbdev-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <55A67FDB.8010602-RbZlAiThDcE@public.gmane.org>
To get full support for parallel and LVDS displays with drm:
Add representation for clock and data enable polarity in drm_display_mode
flags (similar to HSYNC/VSYNC polarity) and update conversion functions
from/to videomode accordingly.
This is especially important for embedded devices where parallel(RGB) and
LVDS displays are still widely used and drm already plays an important
role.
Tested on Freescale i.MX53(parallel) and i.MX6(LVDS).
Background:
There was the ability to set polarity of clock and data enable signals
in devicetree(display-timing), struct display_timing and struct videomode,
but there was no representation for this in struct drm_display_mode.
Example on Freescale i.MX53/i.MX6 SoC's:
* A parallel display using different clock polarity is set up using
display-timing in devicetree
* ipuv3 parallel outputs clock with wrong polarity
Signed-off-by: Manfred Schlaegl <manfred.schlaegl-RbZlAiThDcE@public.gmane.org>
---
drivers/gpu/drm/drm_modes.c | 16 ++++++++++++++++
include/uapi/drm/drm_mode.h | 5 +++++
2 files changed, 21 insertions(+)
diff --git a/drivers/gpu/drm/drm_modes.c b/drivers/gpu/drm/drm_modes.c
index cd74a09..dbb28b7 100644
--- a/drivers/gpu/drm/drm_modes.c
+++ b/drivers/gpu/drm/drm_modes.c
@@ -605,6 +605,14 @@ void drm_display_mode_from_videomode(const struct videomode *vm,
dmode->flags |= DRM_MODE_FLAG_PVSYNC;
else if (vm->flags & DISPLAY_FLAGS_VSYNC_LOW)
dmode->flags |= DRM_MODE_FLAG_NVSYNC;
+ if (vm->flags & DISPLAY_FLAGS_DE_HIGH)
+ dmode->flags |= DRM_MODE_FLAG_PDE;
+ else if (vm->flags & DISPLAY_FLAGS_DE_LOW)
+ dmode->flags |= DRM_MODE_FLAG_NDE;
+ if (vm->flags & DISPLAY_FLAGS_PIXDATA_POSEDGE)
+ dmode->flags |= DRM_MODE_FLAG_PPIXDATA;
+ else if (vm->flags & DISPLAY_FLAGS_PIXDATA_NEGEDGE)
+ dmode->flags |= DRM_MODE_FLAG_NPIXDATA;
if (vm->flags & DISPLAY_FLAGS_INTERLACED)
dmode->flags |= DRM_MODE_FLAG_INTERLACE;
if (vm->flags & DISPLAY_FLAGS_DOUBLESCAN)
@@ -646,6 +654,14 @@ void drm_display_mode_to_videomode(const struct drm_display_mode *dmode,
vm->flags |= DISPLAY_FLAGS_VSYNC_HIGH;
else if (dmode->flags & DRM_MODE_FLAG_NVSYNC)
vm->flags |= DISPLAY_FLAGS_VSYNC_LOW;
+ if (dmode->flags & DRM_MODE_FLAG_PDE)
+ vm->flags |= DISPLAY_FLAGS_DE_HIGH;
+ else if (dmode->flags & DRM_MODE_FLAG_NDE)
+ vm->flags |= DISPLAY_FLAGS_DE_LOW;
+ if (dmode->flags & DRM_MODE_FLAG_PPIXDATA)
+ vm->flags |= DISPLAY_FLAGS_PIXDATA_POSEDGE;
+ else if (dmode->flags & DRM_MODE_FLAG_NPIXDATA)
+ vm->flags |= DISPLAY_FLAGS_PIXDATA_NEGEDGE;
if (dmode->flags & DRM_MODE_FLAG_INTERLACE)
vm->flags |= DISPLAY_FLAGS_INTERLACED;
if (dmode->flags & DRM_MODE_FLAG_DBLSCAN)
diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
index 359107a..cb4912b 100644
--- a/include/uapi/drm/drm_mode.h
+++ b/include/uapi/drm/drm_mode.h
@@ -72,6 +72,11 @@
#define DRM_MODE_FLAG_3D_L_DEPTH_GFX_GFX_DEPTH (6<<14)
#define DRM_MODE_FLAG_3D_TOP_AND_BOTTOM (7<<14)
#define DRM_MODE_FLAG_3D_SIDE_BY_SIDE_HALF (8<<14)
+/* flags for display data enable and clock polarity */
+#define DRM_MODE_FLAG_PDE (1<<19)
+#define DRM_MODE_FLAG_NDE (1<<20)
+#define DRM_MODE_FLAG_PPIXDATA (1<<21)
+#define DRM_MODE_FLAG_NPIXDATA (1<<22)
/* DPMS flags */
--
2.1.4
^ permalink raw reply related
* [RFC PATCH 0/2] drm: add support for for clk and de polarity & gpu: ipu-v3: use clock and de polarity
From: Manfred Schlaegl @ 2015-07-15 15:44 UTC (permalink / raw)
To: David Airlie, Philipp Zabel, Jean-Christophe Plagniol-Villard,
Tomi Valkeinen
Cc: dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA, Manfred Schlaegl,
Steve Longerbeam, Deepak Das, Jiada Wang,
linux-fbdev-u79uwXL29TY76Z2rM5mHXA
Hello!
These patches address a problem we ran into using parallel displays
with Freescale i.MX53 and i.MX6 SoC's.
In short: We wanted to change the clock signal polarity by using
display-timing in the devicetree description, but the output signal stayed
unchanged.
Parallel displays may have different polarities for clock and data enable
signals. (Clock polarity is also a topic on LVDS displays). This is the
reason why there are some properties in devicetree display-timing [1]
called
* pixelclk-active .. pixel clock polarity
* de-active .. data enable pulse polarity
This properties are correctly represented in struct display_timings [2]
and struct videomode [3] by using enum display_flags [3].
But when it comes to struct drm_display_mode [4] there are no
representations for this.
The properties are "lost in conversion" and never reach imx drm. (or
other drm drivers).
Changing this would be especially important for embedded devices where
parallel(RGB) and LVDS displays are still widely used and drm already
plays an important role.
Following two patches will
1. Introduce representation of clock and data enable polarities
in struct drm_display_mode analog to hsync/vsync signals.
* "drm: add support for for clk and de polarity"
2. Implicitly enable usage of the newly introduced flags in
imx drm (with respect to devicetree compatibility.)
* "gpu: ipu-v3: use clock and de polarity from videomode"
I'm aware that introducing new flags in struct drm_display_mode affects
user space too and therefore has to be considered carefully.
As I'm not really deep in drm user space your suggestions are very
welcome.
best regards,
manfred
[1] Documentation/devicetree/bindings/video/display-timing.txt
[2] include/video/display_timing.h
[3] include/video/videomode.h
[4] include/drm/drm_modes.h
^ permalink raw reply
* Re: KDBUS breaks 'make installmandocs'
From: David Herrmann @ 2015-07-15 14:19 UTC (permalink / raw)
To: Jim Davis
Cc: Ulf Magnusson, Greg KH, Linux API, linux-kernel, Daniel Mack,
David Herrmann, Djalal Harouni
In-Reply-To: <CA+r1ZhjcOvHUcAMbhG4pqrxGMqAXV38Nf8jC_OS-ZxbsFrEj3g@mail.gmail.com>
Hi
On Tue, Jul 14, 2015 at 8:24 PM, Jim Davis <jim.epost@gmail.com> wrote:
> On Mon, Jul 13, 2015 at 7:00 PM, Ulf Magnusson <ulfalizer.lkml@gmail.com> wrote:
>> On Mon, Jul 13, 2015 at 05:39:25PM -0700, Greg KH wrote:
>>> On Sun, Jul 12, 2015 at 08:08:46PM +0200, Ulf Magnusson wrote:
>>> > Hello,
>>> >
>>> > Tested with next-20150709.
>>> >
>>> > Documentation/kdbus/Makefile lacks an 'installmandocs' target, causing
>>> > problems for
>>> >
>>> > %docs: scripts_basic FORCE
>>> > $(Q)$(MAKE) $(build)=scripts build_docproc
>>> > $(Q)$(MAKE) $(build)=Documentation/DocBook $@
>>> > $(Q)$(MAKE) $(build)=Documentation/kdbus $@
>>> >
>>> > from the top-level Makefile. It still works, but no KDBUS-related man
>>> > pages are installed, and the final error isn't reassuring.
>>>
>>> What's the final error?
>>>
>>
>> $ sudo make installmandocs
>> GEN Documentation/DocBook//v4l2.xml
>> find Documentation/DocBook/man -name '*.9' | xargs gzip -f
>> mkdir -p /usr/local/man/man9/
>> install Documentation/DocBook/man/*.9.gz /usr/local/man/man9/
>> make[1]: *** No rule to make target 'installmandocs'. Stop.
>> Makefile:1359: recipe for target 'installmandocs' failed
>> make: *** [installmandocs] Error 2
>>
>>
>> The "*** No rule to make target 'installmandocs'. Stop." error is
>> generated for Documentation/kdbus/Makefile. The other man pages still
>> install fine.
>
> There's a similar problem with 'make cleandocs'. Adding noop rules
> for installdocs and cleandocs to Documentation/kdbus/Makefile at least
> gets rid of the error messages.
So the Docbook targets support much more %docs targets than just
cleandocs and installdocs. I don't think it's reasonable to guarantee
that all those targets are supported for all documentation subdirs.
Hence, I'd propose the attached patch. This should at least make sure
the targets won't fail.
But I think we should look into cleandocs and installmandocs. I'll
have a look into that later. This patch should just fix all the other
targets right now.
Thanks
David
diff --git a/Documentation/kdbus/Makefile b/Documentation/kdbus/Makefile
index af87641..8caffe5 100644
--- a/Documentation/kdbus/Makefile
+++ b/Documentation/kdbus/Makefile
@@ -38,3 +38,7 @@ mandocs: $(MANFILES)
htmldocs: $(HTMLFILES)
clean-files := $(MANFILES) $(HTMLFILES)
+
+# we don't support other %docs targets right now
+%docs:
+ @true
^ permalink raw reply related
* [PATCH -mm v8 7/7] proc: export idle flag via kpageflags
From: Vladimir Davydov @ 2015-07-15 13:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Andres Lagar-Cavilla, Minchan Kim, Raghavendra K T,
Johannes Weiner, Michal Hocko, Greg Thelen, Michel Lespinasse,
David Rientjes, Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet,
linux-api, linux-doc, linux-mm, cgroups, linux-kernel
In-Reply-To: <cover.1436967694.git.vdavydov@parallels.com>
As noted by Minchan, a benefit of reading idle flag from
/proc/kpageflags is that one can easily filter dirty and/or unevictable
pages while estimating the size of unused memory.
Note that idle flag read from /proc/kpageflags may be stale in case the
page was accessed via a PTE, because it would be too costly to iterate
over all page mappings on each /proc/kpageflags read to provide an
up-to-date value. To make sure the flag is up-to-date one has to read
/proc/kpageidle first.
Signed-off-by: Vladimir Davydov <vdavydov@parallels.com>
---
Documentation/vm/pagemap.txt | 6 ++++++
fs/proc/page.c | 3 +++
include/uapi/linux/kernel-page-flags.h | 1 +
3 files changed, 10 insertions(+)
diff --git a/Documentation/vm/pagemap.txt b/Documentation/vm/pagemap.txt
index c9266340852c..5896b7d7fd74 100644
--- a/Documentation/vm/pagemap.txt
+++ b/Documentation/vm/pagemap.txt
@@ -64,6 +64,7 @@ There are five components to pagemap:
22. THP
23. BALLOON
24. ZERO_PAGE
+ 25. IDLE
* /proc/kpagecgroup. This file contains a 64-bit inode number of the
memory cgroup each page is charged to, indexed by PFN. Only available when
@@ -124,6 +125,11 @@ Short descriptions to the page flags:
24. ZERO_PAGE
zero page for pfn_zero or huge_zero page
+25. IDLE
+ page has not been accessed since it was marked idle (see /proc/kpageidle)
+ Note that this flag may be stale in case the page was accessed via a PTE.
+ To make sure the flag is up-to-date one has to read /proc/kpageidle first.
+
[IO related page flags]
1. ERROR IO error occurred
3. UPTODATE page has up-to-date data
diff --git a/fs/proc/page.c b/fs/proc/page.c
index 273537885ab4..13dcb823fe4e 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -150,6 +150,9 @@ u64 stable_page_flags(struct page *page)
if (PageBalloon(page))
u |= 1 << KPF_BALLOON;
+ if (page_is_idle(page))
+ u |= 1 << KPF_IDLE;
+
u |= kpf_copy_bit(k, KPF_LOCKED, PG_locked);
u |= kpf_copy_bit(k, KPF_SLAB, PG_slab);
diff --git a/include/uapi/linux/kernel-page-flags.h b/include/uapi/linux/kernel-page-flags.h
index a6c4962e5d46..5da5f8751ce7 100644
--- a/include/uapi/linux/kernel-page-flags.h
+++ b/include/uapi/linux/kernel-page-flags.h
@@ -33,6 +33,7 @@
#define KPF_THP 22
#define KPF_BALLOON 23
#define KPF_ZERO_PAGE 24
+#define KPF_IDLE 25
#endif /* _UAPILINUX_KERNEL_PAGE_FLAGS_H */
--
2.1.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH -mm v8 6/7] proc: add kpageidle file
From: Vladimir Davydov @ 2015-07-15 13:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Andres Lagar-Cavilla, Minchan Kim, Raghavendra K T,
Johannes Weiner, Michal Hocko, Greg Thelen, Michel Lespinasse,
David Rientjes, Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet,
linux-api, linux-doc, linux-mm, cgroups, linux-kernel
In-Reply-To: <cover.1436967694.git.vdavydov@parallels.com>
Knowing the portion of memory that is not used by a certain application
or memory cgroup (idle memory) can be useful for partitioning the system
efficiently, e.g. by setting memory cgroup limits appropriately.
Currently, the only means to estimate the amount of idle memory provided
by the kernel is /proc/PID/{clear_refs,smaps}: the user can clear the
access bit for all pages mapped to a particular process by writing 1 to
clear_refs, wait for some time, and then count smaps:Referenced.
However, this method has two serious shortcomings:
- it does not count unmapped file pages
- it affects the reclaimer logic
To overcome these drawbacks, this patch introduces two new page flags,
Idle and Young, and a new proc file, /proc/kpageidle. A page's Idle flag
can only be set from userspace by setting bit in /proc/kpageidle at the
offset corresponding to the page, and it is cleared whenever the page is
accessed either through page tables (it is cleared in page_referenced()
in this case) or using the read(2) system call (mark_page_accessed()).
Thus by setting the Idle flag for pages of a particular workload, which
can be found e.g. by reading /proc/PID/pagemap, waiting for some time to
let the workload access its working set, and then reading the kpageidle
file, one can estimate the amount of pages that are not used by the
workload.
The Young page flag is used to avoid interference with the memory
reclaimer. A page's Young flag is set whenever the Access bit of a page
table entry pointing to the page is cleared by writing to kpageidle. If
page_referenced() is called on a Young page, it will add 1 to its return
value, therefore concealing the fact that the Access bit was cleared.
Note, since there is no room for extra page flags on 32 bit, this
feature uses extended page flags when compiled on 32 bit.
Signed-off-by: Vladimir Davydov <vdavydov@parallels.com>
---
Documentation/vm/pagemap.txt | 12 ++-
fs/proc/page.c | 218 +++++++++++++++++++++++++++++++++++++++++++
fs/proc/task_mmu.c | 4 +-
include/linux/mm.h | 98 +++++++++++++++++++
include/linux/page-flags.h | 11 +++
include/linux/page_ext.h | 4 +
mm/Kconfig | 12 +++
mm/debug.c | 4 +
mm/huge_memory.c | 11 ++-
mm/migrate.c | 5 +
mm/page_ext.c | 3 +
mm/rmap.c | 5 +
mm/swap.c | 2 +
13 files changed, 385 insertions(+), 4 deletions(-)
diff --git a/Documentation/vm/pagemap.txt b/Documentation/vm/pagemap.txt
index a9b7afc8fbc6..c9266340852c 100644
--- a/Documentation/vm/pagemap.txt
+++ b/Documentation/vm/pagemap.txt
@@ -5,7 +5,7 @@ pagemap is a new (as of 2.6.25) set of interfaces in the kernel that allow
userspace programs to examine the page tables and related information by
reading files in /proc.
-There are four components to pagemap:
+There are five components to pagemap:
* /proc/pid/pagemap. This file lets a userspace process find out which
physical frame each virtual page is mapped to. It contains one 64-bit
@@ -69,6 +69,16 @@ There are four components to pagemap:
memory cgroup each page is charged to, indexed by PFN. Only available when
CONFIG_MEMCG is set.
+ * /proc/kpageidle. This file implements a bitmap where each bit corresponds
+ to a page, indexed by PFN. When the bit is set, the corresponding page is
+ idle. A page is considered idle if it has not been accessed since it was
+ marked idle. To mark a page idle one should set the bit corresponding to the
+ page by writing to the file. A value written to the file is OR-ed with the
+ current bitmap value. Only user memory pages can be marked idle, for other
+ page types input is silently ignored. Writing to this file beyond max PFN
+ results in the ENXIO error. Only available when CONFIG_IDLE_PAGE_TRACKING is
+ set.
+
Short descriptions to the page flags:
0. LOCKED
diff --git a/fs/proc/page.c b/fs/proc/page.c
index 70d23245dd43..273537885ab4 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -5,6 +5,8 @@
#include <linux/ksm.h>
#include <linux/mm.h>
#include <linux/mmzone.h>
+#include <linux/rmap.h>
+#include <linux/mmu_notifier.h>
#include <linux/huge_mm.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
@@ -16,6 +18,7 @@
#define KPMSIZE sizeof(u64)
#define KPMMASK (KPMSIZE - 1)
+#define KPMBITS (KPMSIZE * BITS_PER_BYTE)
/* /proc/kpagecount - an array exposing page counts
*
@@ -275,6 +278,217 @@ static const struct file_operations proc_kpagecgroup_operations = {
};
#endif /* CONFIG_MEMCG */
+#ifdef CONFIG_IDLE_PAGE_TRACKING
+/*
+ * Idle page tracking only considers user memory pages, for other types of
+ * pages the idle flag is always unset and an attempt to set it is silently
+ * ignored.
+ *
+ * We treat a page as a user memory page if it is on an LRU list, because it is
+ * always safe to pass such a page to rmap_walk(), which is essential for idle
+ * page tracking. With such an indicator of user pages we can skip isolated
+ * pages, but since there are not usually many of them, it will hardly affect
+ * the overall result.
+ *
+ * This function tries to get a user memory page by pfn as described above.
+ */
+static struct page *kpageidle_get_page(unsigned long pfn)
+{
+ struct page *page;
+ struct zone *zone;
+
+ if (!pfn_valid(pfn))
+ return NULL;
+
+ page = pfn_to_page(pfn);
+ if (!page || !PageLRU(page) ||
+ !get_page_unless_zero(page))
+ return NULL;
+
+ zone = page_zone(page);
+ spin_lock_irq(&zone->lru_lock);
+ if (unlikely(!PageLRU(page))) {
+ put_page(page);
+ page = NULL;
+ }
+ spin_unlock_irq(&zone->lru_lock);
+ return page;
+}
+
+static int kpageidle_clear_pte_refs_one(struct page *page,
+ struct vm_area_struct *vma,
+ unsigned long addr, void *arg)
+{
+ struct mm_struct *mm = vma->vm_mm;
+ spinlock_t *ptl;
+ pmd_t *pmd;
+ pte_t *pte;
+ bool referenced = false;
+
+ if (unlikely(PageTransHuge(page))) {
+ pmd = page_check_address_pmd(page, mm, addr,
+ PAGE_CHECK_ADDRESS_PMD_FLAG, &ptl);
+ if (pmd) {
+ referenced = pmdp_clear_young_notify(vma, addr, pmd);
+ spin_unlock(ptl);
+ }
+ } else {
+ pte = page_check_address(page, mm, addr, &ptl, 0);
+ if (pte) {
+ referenced = ptep_clear_young_notify(vma, addr, pte);
+ pte_unmap_unlock(pte, ptl);
+ }
+ }
+ if (referenced) {
+ clear_page_idle(page);
+ /*
+ * We cleared the referenced bit in a mapping to this page. To
+ * avoid interference with page reclaim, mark it young so that
+ * page_referenced() will return > 0.
+ */
+ set_page_young(page);
+ }
+ return SWAP_AGAIN;
+}
+
+static void kpageidle_clear_pte_refs(struct page *page)
+{
+ struct rmap_walk_control rwc = {
+ .rmap_one = kpageidle_clear_pte_refs_one,
+ .anon_lock = page_lock_anon_vma_read,
+ };
+ bool need_lock;
+
+ if (!page_mapped(page) ||
+ !page_rmapping(page))
+ return;
+
+ need_lock = !PageAnon(page) || PageKsm(page);
+ if (need_lock && !trylock_page(page))
+ return;
+
+ rmap_walk(page, &rwc);
+
+ if (need_lock)
+ unlock_page(page);
+}
+
+static ssize_t kpageidle_read(struct file *file, char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ u64 __user *out = (u64 __user *)buf;
+ struct page *page;
+ unsigned long pfn, end_pfn;
+ ssize_t ret = 0;
+ u64 idle_bitmap = 0;
+ int bit;
+
+ if (*ppos & KPMMASK || count & KPMMASK)
+ return -EINVAL;
+
+ pfn = *ppos * BITS_PER_BYTE;
+ if (pfn >= max_pfn)
+ return 0;
+
+ end_pfn = pfn + count * BITS_PER_BYTE;
+ if (end_pfn > max_pfn)
+ end_pfn = ALIGN(max_pfn, KPMBITS);
+
+ for (; pfn < end_pfn; pfn++) {
+ bit = pfn % KPMBITS;
+ page = kpageidle_get_page(pfn);
+ if (page) {
+ if (page_is_idle(page)) {
+ /*
+ * The page might have been referenced via a
+ * pte, in which case it is not idle. Clear
+ * refs and recheck.
+ */
+ kpageidle_clear_pte_refs(page);
+ if (page_is_idle(page))
+ idle_bitmap |= 1ULL << bit;
+ }
+ put_page(page);
+ }
+ if (bit == KPMBITS - 1) {
+ if (put_user(idle_bitmap, out)) {
+ ret = -EFAULT;
+ break;
+ }
+ idle_bitmap = 0;
+ out++;
+ }
+ }
+
+ *ppos += (char __user *)out - buf;
+ if (!ret)
+ ret = (char __user *)out - buf;
+ return ret;
+}
+
+static ssize_t kpageidle_write(struct file *file, const char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ const u64 __user *in = (const u64 __user *)buf;
+ struct page *page;
+ unsigned long pfn, end_pfn;
+ ssize_t ret = 0;
+ u64 idle_bitmap = 0;
+ int bit;
+
+ if (*ppos & KPMMASK || count & KPMMASK)
+ return -EINVAL;
+
+ pfn = *ppos * BITS_PER_BYTE;
+ if (pfn >= max_pfn)
+ return -ENXIO;
+
+ end_pfn = pfn + count * BITS_PER_BYTE;
+ if (end_pfn > max_pfn)
+ end_pfn = ALIGN(max_pfn, KPMBITS);
+
+ for (; pfn < end_pfn; pfn++) {
+ bit = pfn % KPMBITS;
+ if (bit == 0) {
+ if (get_user(idle_bitmap, in)) {
+ ret = -EFAULT;
+ break;
+ }
+ in++;
+ }
+ if (idle_bitmap >> bit & 1) {
+ page = kpageidle_get_page(pfn);
+ if (page) {
+ kpageidle_clear_pte_refs(page);
+ set_page_idle(page);
+ put_page(page);
+ }
+ }
+ }
+
+ *ppos += (const char __user *)in - buf;
+ if (!ret)
+ ret = (const char __user *)in - buf;
+ return ret;
+}
+
+static const struct file_operations proc_kpageidle_operations = {
+ .llseek = mem_lseek,
+ .read = kpageidle_read,
+ .write = kpageidle_write,
+};
+
+#ifndef CONFIG_64BIT
+static bool need_page_idle(void)
+{
+ return true;
+}
+struct page_ext_operations page_idle_ops = {
+ .need = need_page_idle,
+};
+#endif
+#endif /* CONFIG_IDLE_PAGE_TRACKING */
+
static int __init proc_page_init(void)
{
proc_create("kpagecount", S_IRUSR, NULL, &proc_kpagecount_operations);
@@ -282,6 +496,10 @@ static int __init proc_page_init(void)
#ifdef CONFIG_MEMCG
proc_create("kpagecgroup", S_IRUSR, NULL, &proc_kpagecgroup_operations);
#endif
+#ifdef CONFIG_IDLE_PAGE_TRACKING
+ proc_create("kpageidle", S_IRUSR | S_IWUSR, NULL,
+ &proc_kpageidle_operations);
+#endif
return 0;
}
fs_initcall(proc_page_init);
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 3b4d8255e806..3efd7f641f92 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -458,7 +458,7 @@ static void smaps_account(struct mem_size_stats *mss, struct page *page,
mss->resident += size;
/* Accumulate the size in pages that have been accessed. */
- if (young || PageReferenced(page))
+ if (young || page_is_young(page) || PageReferenced(page))
mss->referenced += size;
mapcount = page_mapcount(page);
if (mapcount >= 2) {
@@ -810,6 +810,7 @@ static int clear_refs_pte_range(pmd_t *pmd, unsigned long addr,
/* Clear accessed and referenced bits. */
pmdp_test_and_clear_young(vma, addr, pmd);
+ test_and_clear_page_young(page);
ClearPageReferenced(page);
out:
spin_unlock(ptl);
@@ -837,6 +838,7 @@ out:
/* Clear accessed and referenced bits. */
ptep_test_and_clear_young(vma, addr, pte);
+ test_and_clear_page_young(page);
ClearPageReferenced(page);
}
pte_unmap_unlock(pte - 1, ptl);
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 7f471789781a..de450c1191b9 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2205,5 +2205,103 @@ void __init setup_nr_node_ids(void);
static inline void setup_nr_node_ids(void) {}
#endif
+#ifdef CONFIG_IDLE_PAGE_TRACKING
+#ifdef CONFIG_64BIT
+static inline bool page_is_young(struct page *page)
+{
+ return PageYoung(page);
+}
+
+static inline void set_page_young(struct page *page)
+{
+ SetPageYoung(page);
+}
+
+static inline bool test_and_clear_page_young(struct page *page)
+{
+ return TestClearPageYoung(page);
+}
+
+static inline bool page_is_idle(struct page *page)
+{
+ return PageIdle(page);
+}
+
+static inline void set_page_idle(struct page *page)
+{
+ SetPageIdle(page);
+}
+
+static inline void clear_page_idle(struct page *page)
+{
+ ClearPageIdle(page);
+}
+#else /* !CONFIG_64BIT */
+/*
+ * If there is not enough space to store Idle and Young bits in page flags, use
+ * page ext flags instead.
+ */
+extern struct page_ext_operations page_idle_ops;
+
+static inline bool page_is_young(struct page *page)
+{
+ return test_bit(PAGE_EXT_YOUNG, &lookup_page_ext(page)->flags);
+}
+
+static inline void set_page_young(struct page *page)
+{
+ set_bit(PAGE_EXT_YOUNG, &lookup_page_ext(page)->flags);
+}
+
+static inline bool test_and_clear_page_young(struct page *page)
+{
+ return test_and_clear_bit(PAGE_EXT_YOUNG,
+ &lookup_page_ext(page)->flags);
+}
+
+static inline bool page_is_idle(struct page *page)
+{
+ return test_bit(PAGE_EXT_IDLE, &lookup_page_ext(page)->flags);
+}
+
+static inline void set_page_idle(struct page *page)
+{
+ set_bit(PAGE_EXT_IDLE, &lookup_page_ext(page)->flags);
+}
+
+static inline void clear_page_idle(struct page *page)
+{
+ clear_bit(PAGE_EXT_IDLE, &lookup_page_ext(page)->flags);
+}
+#endif /* CONFIG_64BIT */
+#else /* !CONFIG_IDLE_PAGE_TRACKING */
+static inline bool page_is_young(struct page *page)
+{
+ return false;
+}
+
+static inline void set_page_young(struct page *page)
+{
+}
+
+static inline bool test_and_clear_page_young(struct page *page)
+{
+ return false;
+}
+
+static inline bool page_is_idle(struct page *page)
+{
+ return false;
+}
+
+static inline void set_page_idle(struct page *page)
+{
+}
+
+static inline void clear_page_idle(struct page *page)
+{
+}
+#endif /* CONFIG_IDLE_PAGE_TRACKING */
+
#endif /* __KERNEL__ */
#endif /* _LINUX_MM_H */
diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index 91b7f9b2b774..478f2241f284 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -109,6 +109,10 @@ enum pageflags {
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
PG_compound_lock,
#endif
+#if defined(CONFIG_IDLE_PAGE_TRACKING) && defined(CONFIG_64BIT)
+ PG_young,
+ PG_idle,
+#endif
__NR_PAGEFLAGS,
/* Filesystems */
@@ -363,6 +367,13 @@ PAGEFLAG_FALSE(HWPoison)
#define __PG_HWPOISON 0
#endif
+#if defined(CONFIG_IDLE_PAGE_TRACKING) && defined(CONFIG_64BIT)
+TESTPAGEFLAG(Young, young, PF_ANY)
+SETPAGEFLAG(Young, young, PF_ANY)
+TESTCLEARFLAG(Young, young, PF_ANY)
+PAGEFLAG(Idle, idle, PF_ANY)
+#endif
+
/*
* On an anonymous page mapped into a user virtual memory area,
* page->mapping points to its anon_vma, not to a struct address_space;
diff --git a/include/linux/page_ext.h b/include/linux/page_ext.h
index c42981cd99aa..17f118a82854 100644
--- a/include/linux/page_ext.h
+++ b/include/linux/page_ext.h
@@ -26,6 +26,10 @@ enum page_ext_flags {
PAGE_EXT_DEBUG_POISON, /* Page is poisoned */
PAGE_EXT_DEBUG_GUARD,
PAGE_EXT_OWNER,
+#if defined(CONFIG_IDLE_PAGE_TRACKING) && !defined(CONFIG_64BIT)
+ PAGE_EXT_YOUNG,
+ PAGE_EXT_IDLE,
+#endif
};
/*
diff --git a/mm/Kconfig b/mm/Kconfig
index e79de2bd12cd..db817e2c2ec8 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -654,3 +654,15 @@ config DEFERRED_STRUCT_PAGE_INIT
when kswapd starts. This has a potential performance impact on
processes running early in the lifetime of the systemm until kswapd
finishes the initialisation.
+
+config IDLE_PAGE_TRACKING
+ bool "Enable idle page tracking"
+ select PROC_PAGE_MONITOR
+ select PAGE_EXTENSION if !64BIT
+ help
+ This feature allows to estimate the amount of user pages that have
+ not been touched during a given period of time. This information can
+ be useful to tune memory cgroup limits and/or for job placement
+ within a compute cluster.
+
+ See Documentation/vm/pagemap.txt for more details.
diff --git a/mm/debug.c b/mm/debug.c
index 76089ddf99ea..6c1b3ea61bfd 100644
--- a/mm/debug.c
+++ b/mm/debug.c
@@ -48,6 +48,10 @@ static const struct trace_print_flags pageflag_names[] = {
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
{1UL << PG_compound_lock, "compound_lock" },
#endif
+#if defined(CONFIG_IDLE_PAGE_TRACKING) && defined(CONFIG_64BIT)
+ {1UL << PG_young, "young" },
+ {1UL << PG_idle, "idle" },
+#endif
};
static void dump_flags(unsigned long flags,
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 9671f51e954d..bb6d2ec1f268 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1754,6 +1754,11 @@ static void __split_huge_page_refcount(struct page *page,
/* clear PageTail before overwriting first_page */
smp_wmb();
+ if (page_is_young(page))
+ set_page_young(page_tail);
+ if (page_is_idle(page))
+ set_page_idle(page_tail);
+
/*
* __split_huge_page_splitting() already set the
* splitting bit in all pmd that could map this
@@ -2259,7 +2264,8 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma,
VM_BUG_ON_PAGE(PageLRU(page), page);
/* If there is no mapped pte young don't collapse the page */
- if (pte_young(pteval) || PageReferenced(page) ||
+ if (pte_young(pteval) ||
+ page_is_young(page) || PageReferenced(page) ||
mmu_notifier_test_young(vma->vm_mm, address))
referenced = true;
}
@@ -2686,7 +2692,8 @@ static int khugepaged_scan_pmd(struct mm_struct *mm,
*/
if (page_count(page) != 1 + !!PageSwapCache(page))
goto out_unmap;
- if (pte_young(pteval) || PageReferenced(page) ||
+ if (pte_young(pteval) ||
+ page_is_young(page) || PageReferenced(page) ||
mmu_notifier_test_young(vma->vm_mm, address))
referenced = true;
}
diff --git a/mm/migrate.c b/mm/migrate.c
index 236ee25e79d9..3e7bb4f2b51c 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -524,6 +524,11 @@ void migrate_page_copy(struct page *newpage, struct page *page)
__set_page_dirty_nobuffers(newpage);
}
+ if (page_is_young(page))
+ set_page_young(newpage);
+ if (page_is_idle(page))
+ set_page_idle(newpage);
+
/*
* Copy NUMA information to the new page, to prevent over-eager
* future migrations of this same page.
diff --git a/mm/page_ext.c b/mm/page_ext.c
index d86fd2f5353f..e4b3af054bf2 100644
--- a/mm/page_ext.c
+++ b/mm/page_ext.c
@@ -59,6 +59,9 @@ static struct page_ext_operations *page_ext_ops[] = {
#ifdef CONFIG_PAGE_OWNER
&page_owner_ops,
#endif
+#if defined(CONFIG_IDLE_PAGE_TRACKING) && !defined(CONFIG_64BIT)
+ &page_idle_ops,
+#endif
};
static unsigned long total_usage;
diff --git a/mm/rmap.c b/mm/rmap.c
index 49b244b1f18c..c96677ade3d1 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -798,6 +798,11 @@ static int page_referenced_one(struct page *page, struct vm_area_struct *vma,
pte_unmap_unlock(pte, ptl);
}
+ if (referenced)
+ clear_page_idle(page);
+ if (test_and_clear_page_young(page))
+ referenced++;
+
if (referenced) {
pra->referenced++;
pra->vm_flags |= vma->vm_flags;
diff --git a/mm/swap.c b/mm/swap.c
index ab7c338eda87..db43c9b4891d 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -623,6 +623,8 @@ void mark_page_accessed(struct page *page)
} else if (!PageReferenced(page)) {
SetPageReferenced(page);
}
+ if (page_is_idle(page))
+ clear_page_idle(page);
}
EXPORT_SYMBOL(mark_page_accessed);
--
2.1.4
^ permalink raw reply related
* [PATCH -mm v8 5/7] mmu-notifier: add clear_young callback
From: Vladimir Davydov @ 2015-07-15 13:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Andres Lagar-Cavilla, Minchan Kim, Raghavendra K T,
Johannes Weiner, Michal Hocko, Greg Thelen, Michel Lespinasse,
David Rientjes, Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet,
linux-api, linux-doc, linux-mm, cgroups, linux-kernel
In-Reply-To: <cover.1436967694.git.vdavydov@parallels.com>
In the scope of the idle memory tracking feature, which is introduced by
the following patch, we need to clear the referenced/accessed bit not
only in primary, but also in secondary ptes. The latter is required in
order to estimate wss of KVM VMs. At the same time we want to avoid
flushing tlb, because it is quite expensive and it won't really affect
the final result.
Currently, there is no function for clearing pte young bit that would
meet our requirements, so this patch introduces one. To achieve that we
have to add a new mmu-notifier callback, clear_young, since there is no
method for testing-and-clearing a secondary pte w/o flushing tlb. The
new method is not mandatory and currently only implemented by KVM.
Signed-off-by: Vladimir Davydov <vdavydov@parallels.com>
---
include/linux/mmu_notifier.h | 44 ++++++++++++++++++++++++++++++++++++++++++++
mm/mmu_notifier.c | 17 +++++++++++++++++
virt/kvm/kvm_main.c | 18 ++++++++++++++++++
3 files changed, 79 insertions(+)
diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h
index 61cd67f4d788..a5b17137c683 100644
--- a/include/linux/mmu_notifier.h
+++ b/include/linux/mmu_notifier.h
@@ -66,6 +66,16 @@ struct mmu_notifier_ops {
unsigned long end);
/*
+ * clear_young is a lightweight version of clear_flush_young. Like the
+ * latter, it is supposed to test-and-clear the young/accessed bitflag
+ * in the secondary pte, but it may omit flushing the secondary tlb.
+ */
+ int (*clear_young)(struct mmu_notifier *mn,
+ struct mm_struct *mm,
+ unsigned long start,
+ unsigned long end);
+
+ /*
* test_young is called to check the young/accessed bitflag in
* the secondary pte. This is used to know if the page is
* frequently used without actually clearing the flag or tearing
@@ -203,6 +213,9 @@ extern void __mmu_notifier_release(struct mm_struct *mm);
extern int __mmu_notifier_clear_flush_young(struct mm_struct *mm,
unsigned long start,
unsigned long end);
+extern int __mmu_notifier_clear_young(struct mm_struct *mm,
+ unsigned long start,
+ unsigned long end);
extern int __mmu_notifier_test_young(struct mm_struct *mm,
unsigned long address);
extern void __mmu_notifier_change_pte(struct mm_struct *mm,
@@ -231,6 +244,15 @@ static inline int mmu_notifier_clear_flush_young(struct mm_struct *mm,
return 0;
}
+static inline int mmu_notifier_clear_young(struct mm_struct *mm,
+ unsigned long start,
+ unsigned long end)
+{
+ if (mm_has_notifiers(mm))
+ return __mmu_notifier_clear_young(mm, start, end);
+ return 0;
+}
+
static inline int mmu_notifier_test_young(struct mm_struct *mm,
unsigned long address)
{
@@ -311,6 +333,28 @@ static inline void mmu_notifier_mm_destroy(struct mm_struct *mm)
__young; \
})
+#define ptep_clear_young_notify(__vma, __address, __ptep) \
+({ \
+ int __young; \
+ struct vm_area_struct *___vma = __vma; \
+ unsigned long ___address = __address; \
+ __young = ptep_test_and_clear_young(___vma, ___address, __ptep);\
+ __young |= mmu_notifier_clear_young(___vma->vm_mm, ___address, \
+ ___address + PAGE_SIZE); \
+ __young; \
+})
+
+#define pmdp_clear_young_notify(__vma, __address, __pmdp) \
+({ \
+ int __young; \
+ struct vm_area_struct *___vma = __vma; \
+ unsigned long ___address = __address; \
+ __young = pmdp_test_and_clear_young(___vma, ___address, __pmdp);\
+ __young |= mmu_notifier_clear_young(___vma->vm_mm, ___address, \
+ ___address + PMD_SIZE); \
+ __young; \
+})
+
#define ptep_clear_flush_notify(__vma, __address, __ptep) \
({ \
unsigned long ___addr = __address & PAGE_MASK; \
diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c
index 3b9b3d0741b2..5fbdd367bbed 100644
--- a/mm/mmu_notifier.c
+++ b/mm/mmu_notifier.c
@@ -123,6 +123,23 @@ int __mmu_notifier_clear_flush_young(struct mm_struct *mm,
return young;
}
+int __mmu_notifier_clear_young(struct mm_struct *mm,
+ unsigned long start,
+ unsigned long end)
+{
+ struct mmu_notifier *mn;
+ int young = 0, id;
+
+ id = srcu_read_lock(&srcu);
+ hlist_for_each_entry_rcu(mn, &mm->mmu_notifier_mm->list, hlist) {
+ if (mn->ops->clear_young)
+ young |= mn->ops->clear_young(mn, mm, start, end);
+ }
+ srcu_read_unlock(&srcu, id);
+
+ return young;
+}
+
int __mmu_notifier_test_young(struct mm_struct *mm,
unsigned long address)
{
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 05148a43ef9c..61500cb028a3 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -388,6 +388,23 @@ static int kvm_mmu_notifier_clear_flush_young(struct mmu_notifier *mn,
return young;
}
+static int kvm_mmu_notifier_clear_young(struct mmu_notifier *mn,
+ struct mm_struct *mm,
+ unsigned long start,
+ unsigned long end)
+{
+ struct kvm *kvm = mmu_notifier_to_kvm(mn);
+ int young, idx;
+
+ idx = srcu_read_lock(&kvm->srcu);
+ spin_lock(&kvm->mmu_lock);
+ young = kvm_age_hva(kvm, start, end);
+ spin_unlock(&kvm->mmu_lock);
+ srcu_read_unlock(&kvm->srcu, idx);
+
+ return young;
+}
+
static int kvm_mmu_notifier_test_young(struct mmu_notifier *mn,
struct mm_struct *mm,
unsigned long address)
@@ -420,6 +437,7 @@ static const struct mmu_notifier_ops kvm_mmu_notifier_ops = {
.invalidate_range_start = kvm_mmu_notifier_invalidate_range_start,
.invalidate_range_end = kvm_mmu_notifier_invalidate_range_end,
.clear_flush_young = kvm_mmu_notifier_clear_flush_young,
+ .clear_young = kvm_mmu_notifier_clear_young,
.test_young = kvm_mmu_notifier_test_young,
.change_pte = kvm_mmu_notifier_change_pte,
.release = kvm_mmu_notifier_release,
--
2.1.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH -mm v8 4/7] proc: add kpagecgroup file
From: Vladimir Davydov @ 2015-07-15 13:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Andres Lagar-Cavilla, Minchan Kim, Raghavendra K T,
Johannes Weiner, Michal Hocko, Greg Thelen, Michel Lespinasse,
David Rientjes, Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet,
linux-api, linux-doc, linux-mm, cgroups, linux-kernel
In-Reply-To: <cover.1436967694.git.vdavydov@parallels.com>
/proc/kpagecgroup contains a 64-bit inode number of the memory cgroup
each page is charged to, indexed by PFN. Having this information is
useful for estimating a cgroup working set size.
The file is present if CONFIG_PROC_PAGE_MONITOR && CONFIG_MEMCG.
Signed-off-by: Vladimir Davydov <vdavydov@parallels.com>
---
Documentation/vm/pagemap.txt | 6 ++++-
fs/proc/page.c | 53 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 58 insertions(+), 1 deletion(-)
diff --git a/Documentation/vm/pagemap.txt b/Documentation/vm/pagemap.txt
index 6bfbc172cdb9..a9b7afc8fbc6 100644
--- a/Documentation/vm/pagemap.txt
+++ b/Documentation/vm/pagemap.txt
@@ -5,7 +5,7 @@ pagemap is a new (as of 2.6.25) set of interfaces in the kernel that allow
userspace programs to examine the page tables and related information by
reading files in /proc.
-There are three components to pagemap:
+There are four components to pagemap:
* /proc/pid/pagemap. This file lets a userspace process find out which
physical frame each virtual page is mapped to. It contains one 64-bit
@@ -65,6 +65,10 @@ There are three components to pagemap:
23. BALLOON
24. ZERO_PAGE
+ * /proc/kpagecgroup. This file contains a 64-bit inode number of the
+ memory cgroup each page is charged to, indexed by PFN. Only available when
+ CONFIG_MEMCG is set.
+
Short descriptions to the page flags:
0. LOCKED
diff --git a/fs/proc/page.c b/fs/proc/page.c
index 7eee2d8b97d9..70d23245dd43 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -9,6 +9,7 @@
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/hugetlb.h>
+#include <linux/memcontrol.h>
#include <linux/kernel-page-flags.h>
#include <asm/uaccess.h>
#include "internal.h"
@@ -225,10 +226,62 @@ static const struct file_operations proc_kpageflags_operations = {
.read = kpageflags_read,
};
+#ifdef CONFIG_MEMCG
+static ssize_t kpagecgroup_read(struct file *file, char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ u64 __user *out = (u64 __user *)buf;
+ struct page *ppage;
+ unsigned long src = *ppos;
+ unsigned long pfn;
+ ssize_t ret = 0;
+ u64 ino;
+
+ pfn = src / KPMSIZE;
+ count = min_t(unsigned long, count, (max_pfn * KPMSIZE) - src);
+ if (src & KPMMASK || count & KPMMASK)
+ return -EINVAL;
+
+ while (count > 0) {
+ if (pfn_valid(pfn))
+ ppage = pfn_to_page(pfn);
+ else
+ ppage = NULL;
+
+ if (ppage)
+ ino = page_cgroup_ino(ppage);
+ else
+ ino = 0;
+
+ if (put_user(ino, out)) {
+ ret = -EFAULT;
+ break;
+ }
+
+ pfn++;
+ out++;
+ count -= KPMSIZE;
+ }
+
+ *ppos += (char __user *)out - buf;
+ if (!ret)
+ ret = (char __user *)out - buf;
+ return ret;
+}
+
+static const struct file_operations proc_kpagecgroup_operations = {
+ .llseek = mem_lseek,
+ .read = kpagecgroup_read,
+};
+#endif /* CONFIG_MEMCG */
+
static int __init proc_page_init(void)
{
proc_create("kpagecount", S_IRUSR, NULL, &proc_kpagecount_operations);
proc_create("kpageflags", S_IRUSR, NULL, &proc_kpageflags_operations);
+#ifdef CONFIG_MEMCG
+ proc_create("kpagecgroup", S_IRUSR, NULL, &proc_kpagecgroup_operations);
+#endif
return 0;
}
fs_initcall(proc_page_init);
--
2.1.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH -mm v8 3/7] memcg: zap try_get_mem_cgroup_from_page
From: Vladimir Davydov @ 2015-07-15 13:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Andres Lagar-Cavilla, Minchan Kim, Raghavendra K T,
Johannes Weiner, Michal Hocko, Greg Thelen, Michel Lespinasse,
David Rientjes, Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet,
linux-api, linux-doc, linux-mm, cgroups, linux-kernel
In-Reply-To: <cover.1436967694.git.vdavydov@parallels.com>
It is only used in mem_cgroup_try_charge, so fold it in and zap it.
Signed-off-by: Vladimir Davydov <vdavydov@parallels.com>
---
include/linux/memcontrol.h | 6 ------
mm/memcontrol.c | 48 ++++++++++++----------------------------------
2 files changed, 12 insertions(+), 42 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 50069abebc3c..635edfe06bac 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -94,7 +94,6 @@ bool mem_cgroup_is_descendant(struct mem_cgroup *memcg,
struct mem_cgroup *root);
bool task_in_mem_cgroup(struct task_struct *task, struct mem_cgroup *memcg);
-extern struct mem_cgroup *try_get_mem_cgroup_from_page(struct page *page);
extern struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p);
extern struct mem_cgroup *parent_mem_cgroup(struct mem_cgroup *memcg);
@@ -259,11 +258,6 @@ static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page,
return &zone->lruvec;
}
-static inline struct mem_cgroup *try_get_mem_cgroup_from_page(struct page *page)
-{
- return NULL;
-}
-
static inline bool mm_match_cgroup(struct mm_struct *mm,
struct mem_cgroup *memcg)
{
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 894dc2169979..fa1447fcba33 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -2378,40 +2378,6 @@ static void cancel_charge(struct mem_cgroup *memcg, unsigned int nr_pages)
css_put_many(&memcg->css, nr_pages);
}
-/*
- * try_get_mem_cgroup_from_page - look up page's memcg association
- * @page: the page
- *
- * Look up, get a css reference, and return the memcg that owns @page.
- *
- * The page must be locked to prevent racing with swap-in and page
- * cache charges. If coming from an unlocked page table, the caller
- * must ensure the page is on the LRU or this can race with charging.
- */
-struct mem_cgroup *try_get_mem_cgroup_from_page(struct page *page)
-{
- struct mem_cgroup *memcg;
- unsigned short id;
- swp_entry_t ent;
-
- VM_BUG_ON_PAGE(!PageLocked(page), page);
-
- memcg = page->mem_cgroup;
- if (memcg) {
- if (!css_tryget_online(&memcg->css))
- memcg = NULL;
- } else if (PageSwapCache(page)) {
- ent.val = page_private(page);
- id = lookup_swap_cgroup_id(ent);
- rcu_read_lock();
- memcg = mem_cgroup_from_id(id);
- if (memcg && !css_tryget_online(&memcg->css))
- memcg = NULL;
- rcu_read_unlock();
- }
- return memcg;
-}
-
static void lock_page_lru(struct page *page, int *isolated)
{
struct zone *zone = page_zone(page);
@@ -5628,8 +5594,20 @@ int mem_cgroup_try_charge(struct page *page, struct mm_struct *mm,
* the page lock, which serializes swap cache removal, which
* in turn serializes uncharging.
*/
+ VM_BUG_ON_PAGE(!PageLocked(page), page);
if (page->mem_cgroup)
goto out;
+
+ if (do_swap_account) {
+ swp_entry_t ent = { .val = page_private(page), };
+ unsigned short id = lookup_swap_cgroup_id(ent);
+
+ rcu_read_lock();
+ memcg = mem_cgroup_from_id(id);
+ if (memcg && !css_tryget_online(&memcg->css))
+ memcg = NULL;
+ rcu_read_unlock();
+ }
}
if (PageTransHuge(page)) {
@@ -5637,8 +5615,6 @@ int mem_cgroup_try_charge(struct page *page, struct mm_struct *mm,
VM_BUG_ON_PAGE(!PageTransHuge(page), page);
}
- if (do_swap_account && PageSwapCache(page))
- memcg = try_get_mem_cgroup_from_page(page);
if (!memcg)
memcg = get_mem_cgroup_from_mm(mm);
--
2.1.4
^ permalink raw reply related
* [PATCH -mm v8 2/7] hwpoison: use page_cgroup_ino for filtering by memcg
From: Vladimir Davydov @ 2015-07-15 13:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Andres Lagar-Cavilla, Minchan Kim, Raghavendra K T,
Johannes Weiner, Michal Hocko, Greg Thelen, Michel Lespinasse,
David Rientjes, Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet,
linux-api, linux-doc, linux-mm, cgroups, linux-kernel
In-Reply-To: <cover.1436967694.git.vdavydov@parallels.com>
Hwpoison allows to filter pages by memory cgroup ino. Currently, it
calls try_get_mem_cgroup_from_page to obtain the cgroup from a page and
then its ino using cgroup_ino, but now we have an apter method for that,
page_cgroup_ino, so use it instead.
Signed-off-by: Vladimir Davydov <vdavydov@parallels.com>
---
mm/hwpoison-inject.c | 5 +----
mm/memory-failure.c | 16 ++--------------
2 files changed, 3 insertions(+), 18 deletions(-)
diff --git a/mm/hwpoison-inject.c b/mm/hwpoison-inject.c
index bf73ac17dad4..5015679014c1 100644
--- a/mm/hwpoison-inject.c
+++ b/mm/hwpoison-inject.c
@@ -45,12 +45,9 @@ static int hwpoison_inject(void *data, u64 val)
/*
* do a racy check with elevated page count, to make sure PG_hwpoison
* will only be set for the targeted owner (or on a free page).
- * We temporarily take page lock for try_get_mem_cgroup_from_page().
* memory_failure() will redo the check reliably inside page lock.
*/
- lock_page(hpage);
err = hwpoison_filter(hpage);
- unlock_page(hpage);
if (err)
goto put_out;
@@ -126,7 +123,7 @@ static int pfn_inject_init(void)
if (!dentry)
goto fail;
-#ifdef CONFIG_MEMCG_SWAP
+#ifdef CONFIG_MEMCG
dentry = debugfs_create_u64("corrupt-filter-memcg", 0600,
hwpoison_dir, &hwpoison_filter_memcg);
if (!dentry)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 1cf7f2988422..97005396a507 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -130,27 +130,15 @@ static int hwpoison_filter_flags(struct page *p)
* can only guarantee that the page either belongs to the memcg tasks, or is
* a freed page.
*/
-#ifdef CONFIG_MEMCG_SWAP
+#ifdef CONFIG_MEMCG
u64 hwpoison_filter_memcg;
EXPORT_SYMBOL_GPL(hwpoison_filter_memcg);
static int hwpoison_filter_task(struct page *p)
{
- struct mem_cgroup *mem;
- struct cgroup_subsys_state *css;
- unsigned long ino;
-
if (!hwpoison_filter_memcg)
return 0;
- mem = try_get_mem_cgroup_from_page(p);
- if (!mem)
- return -EINVAL;
-
- css = mem_cgroup_css(mem);
- ino = cgroup_ino(css->cgroup);
- css_put(css);
-
- if (ino != hwpoison_filter_memcg)
+ if (page_cgroup_ino(p) != hwpoison_filter_memcg)
return -EINVAL;
return 0;
--
2.1.4
^ permalink raw reply related
* [PATCH -mm v8 1/7] memcg: add page_cgroup_ino helper
From: Vladimir Davydov @ 2015-07-15 13:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Andres Lagar-Cavilla, Minchan Kim, Raghavendra K T,
Johannes Weiner, Michal Hocko, Greg Thelen, Michel Lespinasse,
David Rientjes, Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet,
linux-api, linux-doc, linux-mm, cgroups, linux-kernel
In-Reply-To: <cover.1436967694.git.vdavydov@parallels.com>
This function returns the inode number of the closest online ancestor of
the memory cgroup a page is charged to. It is required for exporting
information about which page is charged to which cgroup to userspace,
which will be introduced by a following patch.
Signed-off-by: Vladimir Davydov <vdavydov@parallels.com>
---
include/linux/memcontrol.h | 1 +
mm/memcontrol.c | 23 +++++++++++++++++++++++
2 files changed, 24 insertions(+)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 73b02b0a8f60..50069abebc3c 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -116,6 +116,7 @@ static inline bool mm_match_cgroup(struct mm_struct *mm,
extern struct cgroup_subsys_state *mem_cgroup_css(struct mem_cgroup *memcg);
extern struct cgroup_subsys_state *mem_cgroup_css_from_page(struct page *page);
+extern unsigned long page_cgroup_ino(struct page *page);
struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *,
struct mem_cgroup *,
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index acb93c554f6e..894dc2169979 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -631,6 +631,29 @@ struct cgroup_subsys_state *mem_cgroup_css_from_page(struct page *page)
return &memcg->css;
}
+/**
+ * page_cgroup_ino - return inode number of the memcg a page is charged to
+ * @page: the page
+ *
+ * Look up the closest online ancestor of the memory cgroup @page is charged to
+ * and return its inode number or 0 if @page is not charged to any cgroup. It
+ * is safe to call this function without holding a reference to @page.
+ */
+unsigned long page_cgroup_ino(struct page *page)
+{
+ struct mem_cgroup *memcg;
+ unsigned long ino = 0;
+
+ rcu_read_lock();
+ memcg = READ_ONCE(page->mem_cgroup);
+ while (memcg && !(memcg->css.flags & CSS_ONLINE))
+ memcg = parent_mem_cgroup(memcg);
+ if (memcg)
+ ino = cgroup_ino(memcg->css.cgroup);
+ rcu_read_unlock();
+ return ino;
+}
+
static struct mem_cgroup_per_zone *
mem_cgroup_page_zoneinfo(struct mem_cgroup *memcg, struct page *page)
{
--
2.1.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH -mm v8 0/7] idle memory tracking
From: Vladimir Davydov @ 2015-07-15 13:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Andres Lagar-Cavilla, Minchan Kim, Raghavendra K T,
Johannes Weiner, Michal Hocko, Greg Thelen, Michel Lespinasse,
David Rientjes, Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet,
linux-api, linux-doc, linux-mm, cgroups, linux-kernel
Hi,
This patch set introduces a new user API for tracking user memory pages
that have not been used for a given period of time. The purpose of this
is to provide the userspace with the means of tracking a workload's
working set, i.e. the set of pages that are actively used by the
workload. Knowing the working set size can be useful for partitioning
the system more efficiently, e.g. by tuning memory cgroup limits
appropriately, or for job placement within a compute cluster.
It is based on top of v4.2-rc1-mmotm-2015-07-06-16-25
---- USE CASES ----
The unified cgroup hierarchy has memory.low and memory.high knobs, which
are defined as the low and high boundaries for the workload working set
size. However, the working set size of a workload may be unknown or
change in time. With this patch set, one can periodically estimate the
amount of memory unused by each cgroup and tune their memory.low and
memory.high parameters accordingly, therefore optimizing the overall
memory utilization.
Another use case is balancing workloads within a compute cluster.
Knowing how much memory is not really used by a workload unit may help
take a more optimal decision when considering migrating the unit to
another node within the cluster.
Also, as noted by Minchan, this would be useful for per-process reclaim
(https://lwn.net/Articles/545668/). With idle tracking, we could reclaim idle
pages only by smart user memory manager.
---- USER API ----
The user API consists of two new proc files:
* /proc/kpageidle. This file implements a bitmap where each bit corresponds
to a page, indexed by PFN. When the bit is set, the corresponding page is
idle. A page is considered idle if it has not been accessed since it was
marked idle. To mark a page idle one should set the bit corresponding to the
page by writing to the file. A value written to the file is OR-ed with the
current bitmap value. Only user memory pages can be marked idle, for other
page types input is silently ignored. Writing to this file beyond max PFN
results in the ENXIO error. Only available when CONFIG_IDLE_PAGE_TRACKING is
set.
This file can be used to estimate the amount of pages that are not
used by a particular workload as follows:
1. mark all pages of interest idle by setting corresponding bits in the
/proc/kpageidle bitmap
2. wait until the workload accesses its working set
3. read /proc/kpageidle and count the number of bits set
* /proc/kpagecgroup. This file contains a 64-bit inode number of the
memory cgroup each page is charged to, indexed by PFN. Only available when
CONFIG_MEMCG is set.
This file can be used to find all pages (including unmapped file
pages) accounted to a particular cgroup. Using /proc/kpageidle, one
can then estimate the cgroup working set size.
For an example of using these files for estimating the amount of unused
memory pages per each memory cgroup, please see the script attached
below.
---- REASONING ----
The reason to introduce the new user API instead of using
/proc/PID/{clear_refs,smaps} is that the latter has two serious
drawbacks:
- it does not count unmapped file pages
- it affects the reclaimer logic
The new API attempts to overcome them both. For more details on how it
is achieved, please see the comment to patch 5.
---- CHANGE LOG ----
Changes in v8:
- clear referenced/accessed bit in secondary ptes while accessing
/proc/kpageidle; this is required to estimate wss of KVM VMs (Andres)
- check the young flag when collapsing a huge page
- copy idle/young flags on page migration
Changes in v7:
This iteration addresses Andres's comments to v6:
- do not reuse page_referenced for clearing idle flag, introduce a
separate function instead; this way we won't issue expensive tlb
flushes on /proc/kpageidle read/write
- propagate young/idle flags from head to tail pages on thp split
- skip compound tail pages while reading/writing /proc/kpageidle
- cleanup page_referenced_one
Changes in v6:
- Split the patch introducing page_cgroup_ino helper to ease review.
- Rebase on top of v4.1-rc7-mmotm-2015-06-09-16-55
Changes in v5:
- Fix possible race between kpageidle_clear_pte_refs() and
__page_set_anon_rmap() by checking that a page is on an LRU list
under zone->lru_lock (Minchan).
- Export idle flag via /proc/kpageflags (Minchan).
- Rebase on top of 4.1-rc3.
Changes in v4:
This iteration primarily addresses Minchan's comments to v3:
- Implement /proc/kpageidle as a bitmap instead of using u64 per each page,
because there does not seem to be any future uses for the other 63 bits.
- Do not double-increase pra->referenced in page_referenced_one() if the page
was young and referenced recently.
- Remove the pointless (page_count == 0) check from kpageidle_get_page().
- Rename kpageidle_clear_refs() to kpageidle_clear_pte_refs().
- Improve comments to kpageidle-related functions.
- Rebase on top of 4.1-rc2.
Note it does not address Minchan's concern of possible __page_set_anon_rmap vs
page_referenced race (see https://lkml.org/lkml/2015/5/3/220) since it is still
unclear if this race can really happen (see https://lkml.org/lkml/2015/5/4/160)
Changes in v3:
- Enable CONFIG_IDLE_PAGE_TRACKING for 32 bit. Since this feature
requires two extra page flags and there is no space for them on 32
bit, page ext is used (thanks to Minchan Kim).
- Minor code cleanups and comments improved.
- Rebase on top of 4.1-rc1.
Changes in v2:
- The main difference from v1 is the API change. In v1 the user can
only set the idle flag for all pages at once, and for clearing the
Idle flag on pages accessed via page tables /proc/PID/clear_refs
should be used.
The main drawback of the v1 approach, as noted by Minchan, is that on
big machines setting the idle flag for each pages can result in CPU
bursts, which would be especially frustrating if the user only wanted
to estimate the amount of idle pages for a particular process or VMA.
With the new API a more fine-grained approach is possible: one can
read a process's /proc/PID/pagemap and set/check the Idle flag only
for those pages of the process's address space he or she is
interested in.
Another good point about the v2 API is that it is possible to limit
/proc/kpage* scanning rate when the user wants to estimate the total
number of idle pages, which is unachievable with the v1 approach.
- Make /proc/kpagecgroup return the ino of the closest online ancestor
in case the cgroup a page is charged to is offline.
- Fix /proc/PID/clear_refs not clearing Young page flag.
- Rebase on top of v4.0-rc6-mmotm-2015-04-01-14-54
v7: https://lkml.org/lkml/2015/7/11/119
v6: https://lkml.org/lkml/2015/6/12/301
v5: https://lkml.org/lkml/2015/5/12/449
v4: https://lkml.org/lkml/2015/5/7/580
v3: https://lkml.org/lkml/2015/4/28/224
v2: https://lkml.org/lkml/2015/4/7/260
v1: https://lkml.org/lkml/2015/3/18/794
---- PATCH SET STRUCTURE ----
The patch set is organized as follows:
- patch 1 adds page_cgroup_ino() helper for the sake of
/proc/kpagecgroup and patches 2-3 do related cleanup
- patch 4 adds /proc/kpagecgroup, which reports cgroup ino each page is
charged to
- patch 5 introduces a new mmu notifier callback, clear_young, which is
a lightweight version of clear_flush_young; it is used in patch 6
- patch 6 implements the idle page tracking feature, including the
userspace API, /proc/kpageidle
- patch 7 exports idle flag via /proc/kpageflags
---- SIMILAR WORKS ----
Originally, the patch for tracking idle memory was proposed back in 2011
by Michel Lespinasse (see http://lwn.net/Articles/459269/). The main
difference between Michel's patch and this one is that Michel
implemented a kernel space daemon for estimating idle memory size per
cgroup while this patch only provides the userspace with the minimal API
for doing the job, leaving the rest up to the userspace. However, they
both share the same idea of Idle/Young page flags to avoid affecting the
reclaimer logic.
---- SCRIPT FOR COUNTING IDLE PAGES PER CGROUP ----
#! /usr/bin/python
#
import os
import stat
import errno
import struct
CGROUP_MOUNT = "/sys/fs/cgroup/memory"
BUFSIZE = 8 * 1024 # must be multiple of 8
def get_hugepage_size():
with open("/proc/meminfo", "r") as f:
for s in f:
k, v = s.split(":")
if k == "Hugepagesize":
return int(v.split()[0]) * 1024
PAGE_SIZE = os.sysconf("SC_PAGE_SIZE")
HUGEPAGE_SIZE = get_hugepage_size()
def set_idle():
f = open("/proc/kpageidle", "wb", BUFSIZE)
while True:
try:
f.write(struct.pack("Q", pow(2, 64) - 1))
except IOError as err:
if err.errno == errno.ENXIO:
break
raise
f.close()
def count_idle():
f_flags = open("/proc/kpageflags", "rb", BUFSIZE)
f_cgroup = open("/proc/kpagecgroup", "rb", BUFSIZE)
with open("/proc/kpageidle", "rb", BUFSIZE) as f:
while f.read(BUFSIZE): pass # update idle flag
idlememsz = {}
while True:
s1, s2 = f_flags.read(8), f_cgroup.read(8)
if not s1 or not s2:
break
flags, = struct.unpack('Q', s1)
cgino, = struct.unpack('Q', s2)
unevictable = (flags >> 18) & 1
huge = (flags >> 22) & 1
idle = (flags >> 25) & 1
if idle and not unevictable:
idlememsz[cgino] = idlememsz.get(cgino, 0) + \
(HUGEPAGE_SIZE if huge else PAGE_SIZE)
f_flags.close()
f_cgroup.close()
return idlememsz
if __name__ == "__main__":
print "Setting the idle flag for each page..."
set_idle()
raw_input("Wait until the workload accesses its working set, "
"then press Enter")
print "Counting idle pages..."
idlememsz = count_idle()
for dir, subdirs, files in os.walk(CGROUP_MOUNT):
ino = os.stat(dir)[stat.ST_INO]
print dir + ": " + str(idlememsz.get(ino, 0) / 1024) + " kB"
---- END SCRIPT ----
Comments are more than welcome.
Thanks,
Vladimir Davydov (7):
memcg: add page_cgroup_ino helper
hwpoison: use page_cgroup_ino for filtering by memcg
memcg: zap try_get_mem_cgroup_from_page
proc: add kpagecgroup file
mmu-notifier: add clear_young callback
proc: add kpageidle file
proc: export idle flag via kpageflags
Documentation/vm/pagemap.txt | 22 ++-
fs/proc/page.c | 274 +++++++++++++++++++++++++++++++++
fs/proc/task_mmu.c | 4 +-
include/linux/memcontrol.h | 7 +-
include/linux/mm.h | 98 ++++++++++++
include/linux/mmu_notifier.h | 44 ++++++
include/linux/page-flags.h | 11 ++
include/linux/page_ext.h | 4 +
include/uapi/linux/kernel-page-flags.h | 1 +
mm/Kconfig | 12 ++
mm/debug.c | 4 +
mm/huge_memory.c | 11 +-
mm/hwpoison-inject.c | 5 +-
mm/memcontrol.c | 71 +++++----
mm/memory-failure.c | 16 +-
mm/migrate.c | 5 +
mm/mmu_notifier.c | 17 ++
mm/page_ext.c | 3 +
mm/rmap.c | 5 +
mm/swap.c | 2 +
virt/kvm/kvm_main.c | 18 +++
21 files changed, 570 insertions(+), 64 deletions(-)
--
2.1.4
^ permalink raw reply
* Re: [PATCH 20/24] task_diag: Only add VMAs for thread_group leader
From: Oleg Nesterov @ 2015-07-15 13:31 UTC (permalink / raw)
To: David Ahern
Cc: Andrey Vagin, linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA, Andrew Morton, Cyrill Gorcunov,
Pavel Emelyanov, Roger Luethi, Arnd Bergmann,
Arnaldo Carvalho de Melo, Andy Lutomirski, Pavel Odintsov
In-Reply-To: <55A5BF0F.7090808-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
On 07/14, David Ahern wrote:
>
> On 7/14/15 1:47 PM, Oleg Nesterov wrote:
>>>
>>> if (show_flags & TASK_DIAG_SHOW_VMA) {
>>> + /* if the request is to dump all threads of all processes
>>> + * only show VMAs for group leader.
>>> + */
>>> + if (req->dump_strategy == TASK_DIAG_DUMP_ALL_THREAD &&
>>> + !thread_group_leader(tsk))
>>> + goto done;
>>> +
>>
>> This doesn't look right, group leader can be a zombie with ->mm == NULL,
>
> Seriously?
Yes, the main thread can do sys_exit() / pthread_exit(), although this
is not that common.
> I'll find some way to track whether VMAs have been dumped for
> a pid.
In case I confused you, the thread_group_leader() check and "goto done"
above are fine, just you can't trust tsk->mm, so
>>> if (i >= n)
>>> err = fill_vma(tsk, skb, cb, &progress, show_flags);
>>
>> so this probably needs something like find_lock_task_mm().
>
> ok.
Yes, you just need to (try) find a subthread with ->mm != NULL if !tsk->mm,
find_lock_task_mm() can help.
Oleg.
^ permalink raw reply
* Re: [PATCH v4 1/2] capabilities: Ambient capabilities
From: James Morris @ 2015-07-15 11:13 UTC (permalink / raw)
To: Andy Lutomirski
Cc: Andrew Morton, Serge E. Hallyn, Serge Hallyn, James Morris,
Jarkko Sakkinen, Ted Ts'o, Andrew G. Morgan, Linux API,
Mimi Zohar, Michael Kerrisk, Austin S Hemmelgarn,
linux-security-module, Aaron Jones, Serge Hallyn, LKML,
Markku Savela, Kees Cook, Jonathan Corbet, Christoph Lameter,
Andy Lutomirski
In-Reply-To: <9f660ee708e58892bfa7012b6b9c840298e24544.1436903028.git.luto-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
On Tue, 14 Jul 2015, Andy Lutomirski wrote:
> ===== The proposed change =====
>
> This patch adds a fifth capability mask called the ambient mask
> (pA). pA does what most people expect pI to do.
>
This looks good, and I think it will lead to better overall security
because people will find capabilities easier to use for real-world
scenarios.
Has it had enough security review? This is a significant new behavior
being added to a widely enabled security module.
- James
--
James Morris
<jmorris-gx6/JNMH7DfYtjvyW6yDsg@public.gmane.org>
^ permalink raw reply
* Re: [PATCH v7 3/9] nvmem: Add nvmem_device based consumer apis.
From: Srinivas Kandagatla @ 2015-07-15 8:21 UTC (permalink / raw)
To: Stephen Boyd
Cc: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
Greg Kroah-Hartman, Rob Herring, Kumar Gala, Mark Brown,
s.hauer-bIcnvbaLZ9MEGnE8C9+IrQ, linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-arm-msm-u79uwXL29TY76Z2rM5mHXA, arnd-r2nGTMty4D4,
pantelis.antoniou-OWPKS81ov/FWk0Htik3J/w,
mporter-OWPKS81ov/FWk0Htik3J/w, stefan.wahren-eS4NqCHxEME,
wxt-TNX95d0MmH7DzftRWevZcw
In-Reply-To: <20150714220607.GP30412-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
Thanks Stephen,
On 14/07/15 23:06, Stephen Boyd wrote:
> On 07/10, Srinivas Kandagatla wrote:
>> +static int devm_nvmem_device_match(struct device *dev, void *res, void *data)
>> +{
>> + struct nvmem_device **nvmem = res;
>> +
>> + if (!nvmem || !*nvmem) {
>> + WARN_ON(!nvmem || !*nvmem);
>
> This could be
>
> if (WARN_ON(!nvmem || !*nvmem))
>
Yep, will fix it.
>> + return 0;
>> + }
>> + return *nvmem == data;
>> +}
>> +
>> [..]
>> +
>> +/**
>> + * nvmem_device_write() - Write cell to a given nvmem device
>> + *
>> + * @nvmem: nvmem device to be written to.
>> + * @offset: offset in nvmem device.
>> + * @bytes: number of bytes to write.
>> + * @buf: buffer to be written.
>> + *
>> + * The return value will be an length of bytes written or non zero on failure.
>
> Should say negative value instead of non-zero? Length is
> non-zero already.
>
> General nitpick comment: Kernel-doc allows for a standard return
> syntax.
Ok, I will fix such instances.
>
> Return: length of bytes written or negative value on failure.
>
>> diff --git a/include/linux/nvmem-provider.h b/include/linux/nvmem-provider.h
>> index f589d3b..74eed42 100644
>> --- a/include/linux/nvmem-provider.h
>> +++ b/include/linux/nvmem-provider.h
>> @@ -12,15 +12,9 @@
>> #ifndef _LINUX_NVMEM_PROVIDER_H
>> #define _LINUX_NVMEM_PROVIDER_H
>>
>> -struct nvmem_device;
>> +#include <linux/nvmem-consumer.h>
>>
>> -struct nvmem_cell_info {
>> - const char *name;
>> - int offset;
>> - int bytes;
>> - int bit_offset;
>> - int nbits;
>> -};
>
> Why does this move from provider to consumer? Can't we do put
> this struct in the right place from the beginning?
>
I will take care of it in next version.
--srini
>> +struct nvmem_device;
>
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH RFC] vfs: add a O_NOMTIME flag
From: NeilBrown @ 2015-07-15 4:54 UTC (permalink / raw)
To: Pavel Machek
Cc: NeilBrown, J. Bruce Fields, John Stoffel, Austin S Hemmelgarn,
Kevin Easton, Theodore Ts'o, Sage Weil, Trond Myklebust,
Dave Chinner, Zach Brown, Alexander Viro,
Linux FS-devel Mailing List, Linux Kernel Mailing List,
Linux API Mailing List
In-Reply-To: <20150714131300.GA6804@amd>
On Tue, 14 Jul 2015 15:13:00 +0200 Pavel Machek <pavel-+ZI9xUNit7I@public.gmane.org> wrote:
> Hi!
>
> > BTW When you "swap" to a file the mtime doesn't get updated. No one seems to
> > complain about that. I guess it is a rather narrow use-case though.
>
> Actually yes, I'd like to complain.
>
> It was not swap, it was mount -o loop, but I guess that's the same
> case. Then rsync refused to work on that file... and being on slow ARM
> system it took me a while to figure out WTF is going on.
>
> So yes, we have problems with mtime, and yes, they matter.
> Pavel
Odd...
I assume you mean
mount -o loop /some/file /mountpoint
and then when you write to the filesystem on /mountpoint the mtime
of /some/file doesn't get updated?
I think it should.
drivers/block/loop.c uses vfs_iter_write() to write to a file.
That calls f_op->write_iter which will typically call
generic_file_write_iter() which will call file_update_time() to update
the time stamps.
What filesystem was /some/file on?
I just did some testing on ext4 and it seems to do the right thing
mtime gets updated.
NeilBrown
^ permalink raw reply
* Re: [PATCH 12/24] task_diag: add a new group to get tasks memory mappings (v2)
From: David Ahern @ 2015-07-15 2:02 UTC (permalink / raw)
To: Oleg Nesterov, Andrey Vagin
Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA, Andrew Morton, Cyrill Gorcunov,
Pavel Emelyanov, Roger Luethi, Arnd Bergmann,
Arnaldo Carvalho de Melo, Andy Lutomirski, Pavel Odintsov
In-Reply-To: <20150714180857.GC8088-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
On 7/14/15 2:08 PM, Oleg Nesterov wrote:
> On 07/06, Andrey Vagin wrote:
>>
>> +static int task_vma_num(struct mm_struct *mm)
>> +{
>> + struct vm_area_struct *vma;
>> + int n_vma = 0;
>> +
>> + if (!mm || !atomic_inc_not_zero(&mm->mm_users))
>> + return 0;
>> +
>> + down_read(&mm->mmap_sem);
>> + for (vma = mm->mmap; vma; vma = vma->vm_next, n_vma++)
>> + ;
>> +
>> + up_read(&mm->mmap_sem);
>> + mmput(mm);
>> +
>> + return n_vma;
>> +}
>
> Hmm. How about
>
> int task_vma_num(struct mm_struct *mm)
> {
> return mm->map_count;
> }
>
> ?
makes sense. Thanks for the pointer.
^ permalink raw reply
* Re: [PATCH 20/24] task_diag: Only add VMAs for thread_group leader
From: David Ahern @ 2015-07-15 2:01 UTC (permalink / raw)
To: Oleg Nesterov, Andrey Vagin
Cc: linux-kernel, linux-api, Andrew Morton, Cyrill Gorcunov,
Pavel Emelyanov, Roger Luethi, Arnd Bergmann,
Arnaldo Carvalho de Melo, Andy Lutomirski, Pavel Odintsov
In-Reply-To: <20150714174703.GA8088@redhat.com>
On 7/14/15 1:47 PM, Oleg Nesterov wrote:
> On 07/06, Andrey Vagin wrote:
>>
>> From: David Ahern <dsahern@gmail.com>
>>
>> threads of a process share the same VMAs, so when dumping all threads
>> for all processes only push vma data for group leader.
>
> ...
>
>> @@ -492,6 +493,13 @@ static int task_diag_fill(struct task_struct *tsk, struct sk_buff *skb,
>> }
>>
>> if (show_flags & TASK_DIAG_SHOW_VMA) {
>> + /* if the request is to dump all threads of all processes
>> + * only show VMAs for group leader.
>> + */
>> + if (req->dump_strategy == TASK_DIAG_DUMP_ALL_THREAD &&
>> + !thread_group_leader(tsk))
>> + goto done;
>> +
>
> This doesn't look right, group leader can be a zombie with ->mm == NULL,
Seriously? I'll find some way to track whether VMAs have been dumped for
a pid.
>
>> if (i >= n)
>> err = fill_vma(tsk, skb, cb, &progress, show_flags);
>
> so this probably needs something like find_lock_task_mm().
ok.
Thanks for the review,
David
^ permalink raw reply
* Re: [PATCH] pci_regs: reintroduce PCI_MSIX_FLAGS_BIRMASK
From: Bjorn Helgaas @ 2015-07-14 23:29 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: linux-kernel, Rajat Jain, Chen, Gong, Guenter Roeck,
=?UTF-8?q?Rafa=C5=82=20Mi=C5=82ecki?=, linux-api, linux-pci
In-Reply-To: <20150713181025-mutt-send-email-mst@redhat.com>
On Mon, Jul 13, 2015 at 06:11:38PM +0300, Michael S. Tsirkin wrote:
> On Wed, Jul 01, 2015 at 12:14:10PM +0200, Michael S. Tsirkin wrote:
> > This partially reverts commit 09a2c73ddfc7f173237fc7209a65b34dd5bcb5ed.
> > PCI: Remove unused PCI_MSIX_FLAGS_BIRMASK definition
> >
> > That commit dropped a symbol from an exported header claiming "no one
> > uses it". This isn't how Linux normally approaches userspace API though,
> > and in fact QEMU build fails if trying to use updated headers from linux
> > 3.12 and up.
> >
> > Sure, userspace can be fixed to use the new symbol, but the cost
> > of keeping the old one around is fairly low, too.
> >
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Sorry, I lost the original message somehow, so I can't reply to it. I
applied this by hand to my for-linus branch for v4.2, thanks!
commit c9ddbac9c89110f77cb0fa07e634aaf1194899aa
Author: Michael S. Tsirkin <mst@redhat.com>
Date: Tue Jul 14 18:27:46 2015 -0500
PCI: Restore PCI_MSIX_FLAGS_BIRMASK definition
09a2c73ddfc7 ("PCI: Remove unused PCI_MSIX_FLAGS_BIRMASK definition")
removed PCI_MSIX_FLAGS_BIRMASK from an exported header because it was
unused in the kernel. But that breaks user programs that were using it
(QEMU in particular).
Restore the PCI_MSIX_FLAGS_BIRMASK definition.
[bhelgaas: changelog]
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
CC: stable@vger.kernel.org # v3.13+
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index efe3443..413417f 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -319,6 +319,7 @@
#define PCI_MSIX_PBA 8 /* Pending Bit Array offset */
#define PCI_MSIX_PBA_BIR 0x00000007 /* BAR index */
#define PCI_MSIX_PBA_OFFSET 0xfffffff8 /* Offset into specified BAR */
+#define PCI_MSIX_FLAGS_BIRMASK PCI_MSIX_PBA_BIR /* deprecated */
#define PCI_CAP_MSIX_SIZEOF 12 /* size of MSIX registers */
/* MSI-X Table entry format */
^ permalink raw reply related
* Re: [PATCH v7 3/9] nvmem: Add nvmem_device based consumer apis.
From: Stephen Boyd @ 2015-07-14 22:06 UTC (permalink / raw)
To: Srinivas Kandagatla
Cc: linux-arm-kernel, Greg Kroah-Hartman, Rob Herring, Kumar Gala,
Mark Brown, s.hauer, linux-api, linux-kernel, devicetree,
linux-arm-msm, arnd, pantelis.antoniou, mporter, stefan.wahren,
wxt
In-Reply-To: <1436521505-10779-1-git-send-email-srinivas.kandagatla@linaro.org>
On 07/10, Srinivas Kandagatla wrote:
> +static int devm_nvmem_device_match(struct device *dev, void *res, void *data)
> +{
> + struct nvmem_device **nvmem = res;
> +
> + if (!nvmem || !*nvmem) {
> + WARN_ON(!nvmem || !*nvmem);
This could be
if (WARN_ON(!nvmem || !*nvmem))
> + return 0;
> + }
> + return *nvmem == data;
> +}
> +
> [..]
> +
> +/**
> + * nvmem_device_write() - Write cell to a given nvmem device
> + *
> + * @nvmem: nvmem device to be written to.
> + * @offset: offset in nvmem device.
> + * @bytes: number of bytes to write.
> + * @buf: buffer to be written.
> + *
> + * The return value will be an length of bytes written or non zero on failure.
Should say negative value instead of non-zero? Length is
non-zero already.
General nitpick comment: Kernel-doc allows for a standard return
syntax.
Return: length of bytes written or negative value on failure.
> diff --git a/include/linux/nvmem-provider.h b/include/linux/nvmem-provider.h
> index f589d3b..74eed42 100644
> --- a/include/linux/nvmem-provider.h
> +++ b/include/linux/nvmem-provider.h
> @@ -12,15 +12,9 @@
> #ifndef _LINUX_NVMEM_PROVIDER_H
> #define _LINUX_NVMEM_PROVIDER_H
>
> -struct nvmem_device;
> +#include <linux/nvmem-consumer.h>
>
> -struct nvmem_cell_info {
> - const char *name;
> - int offset;
> - int bytes;
> - int bit_offset;
> - int nbits;
> -};
Why does this move from provider to consumer? Can't we do put
this struct in the right place from the beginning?
> +struct nvmem_device;
--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
^ permalink raw reply
* Re: [PATCH v7 6/9] nvmem: qfprom: Add Qualcomm QFPROM support.
From: Srinivas Kandagatla @ 2015-07-14 22:02 UTC (permalink / raw)
To: Stephen Boyd
Cc: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
Greg Kroah-Hartman, Rob Herring, Kumar Gala, Mark Brown,
s.hauer-bIcnvbaLZ9MEGnE8C9+IrQ, linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-arm-msm-u79uwXL29TY76Z2rM5mHXA, arnd-r2nGTMty4D4,
pantelis.antoniou-OWPKS81ov/FWk0Htik3J/w,
mporter-OWPKS81ov/FWk0Htik3J/w, stefan.wahren-eS4NqCHxEME,
wxt-TNX95d0MmH7DzftRWevZcw
In-Reply-To: <20150714211831.GN30412-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
Thanks Stephen for the comments.
On 14/07/15 22:18, Stephen Boyd wrote:
> On 07/10, Srinivas Kandagatla wrote:
>> This patch adds QFPROM support driver which is used by other drivers
>> like thermal sensor and cpufreq.
>>
>> On MSM parts there are some efuses (called qfprom) these fuses store
>> things like calibration data, speed bins.. etc. Drivers like cpufreq,
>> thermal sensors would read out this data for configuring the driver.
>>
>> Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
>> ---
>
> I didn't see any reply on v6, but at least delay.h was removed.
> Same comments from v6 below.
Sorry I think I forgot to reply you on v6.
>
>> diff --git a/drivers/nvmem/qfprom.c b/drivers/nvmem/qfprom.c
>> new file mode 100644
>> index 0000000..0e22a03
>> --- /dev/null
>> +++ b/drivers/nvmem/qfprom.c
>> @@ -0,0 +1,87 @@
>> +/*
>> + * Copyright (C) 2015 Srinivas Kandagatla <srinivas.kandagatla-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 and
>> + * only version 2 as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> + * GNU General Public License for more details.
>> + */
>> +
>> +#include <linux/device.h>
>> +#include <linux/io.h>
>
> Is this include used?
No, I don't think so, I will remove it.
>
>> +#include <linux/module.h>
>> +#include <linux/nvmem-provider.h>
>> +#include <linux/platform_device.h>
>> +#include <linux/regmap.h>
>> +#include <linux/slab.h>
>> +
>> +static struct regmap_config qfprom_regmap_config = {
>
> Can this be const?
No, regmap config max_register is updated in the probe so we cant have
this as constant.
>
>> + .reg_bits = 32,
>> + .val_bits = 8,
>> + .reg_stride = 1,
>> +};
>> +
>> +static struct nvmem_config econfig = {
>
> Can this be const?
>
Same here, dev pointer is updated in probe so we can't make this constant.
>> + .name = "qfprom",
>> + .owner = THIS_MODULE,
>> +};
>> +
>
^ permalink raw reply
* Re: [PATCH v7 5/9] Documentation: nvmem: add nvmem api level and how-to doc
From: Srinivas Kandagatla @ 2015-07-14 22:00 UTC (permalink / raw)
To: Stephen Boyd
Cc: linux-arm-kernel, Greg Kroah-Hartman, Rob Herring, Kumar Gala,
Mark Brown, s.hauer, linux-api, linux-kernel, devicetree,
linux-arm-msm, arnd, pantelis.antoniou, mporter, stefan.wahren,
wxt
In-Reply-To: <20150714213205.GO30412@codeaurora.org>
On 14/07/15 22:32, Stephen Boyd wrote:
> On 07/10, Srinivas Kandagatla wrote:
>> diff --git a/Documentation/nvmem/nvmem.txt b/Documentation/nvmem/nvmem.txt
>> new file mode 100644
>> index 0000000..b074b71
>> --- /dev/null
>> +++ b/Documentation/nvmem/nvmem.txt
>> @@ -0,0 +1,152 @@
>> + NVMEM SUBSYSTEM
>> + Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
>> +
>> +This document explains the Simple NVMEM Framework along with the APIs provided,
>
> Why is simple and framework capitalized? Is it the "Simple NVMEM
> Framework" or just the "NVMEM" framework?
>
>> +and how-to-use.
>
> how to use it?
yep,
Thanks Stephen,
I will fix all the comments you raised in next version.
>
>> +
>> +1. Introduction
>> +===============
>> +*NVMEM* is the abbreviation for Non Volatile Memory layer. It is used to
>> +retrieve configuration or SOC or Device specific data from a non volatile memories
> ^ ^
> of remove a?
>
>> +like eeprom, efuses and so on.
>> +
>> +Up until now, NVMEM drivers like eeprom were stored in drivers/misc, where they
>
> Up until now will soon be out of date, perhaps say "before this
> framework existed"?
>
>> +all had to duplicate pretty much the same code to register a sysfs file, allow
>> +in-kernel users to access the content of the devices they were driving, etc.
>> +
>> +This was also a problem as far as other in-kernel users were involved, since
>> +the solutions used were pretty much different from on driver to another, there
> ^
> one
>
yep, will fix it.
>> +was a rather big abstraction leak.
>> +
>> +Introduction of this framework aims at solving this. It also introduces DT
>
> This framework aims to solve these problems.
>
>> +representation for consumer devices to go get the data they require (MAC
>> +Addresses, SoC/Revision ID, part numbers, and so on) from the NVMEMs.
>> +This framework is based on regmap, so that most of the abstraction
>> +available in regmap can be reused, across multiple types of buses.
>> +
>> +NVMEM Providers
>> ++++++++++++++++
>> +
>> +NVMEM provider refers to an entity that implements methods to initialize, read
>> +and write the non-volatile memory.
>> +
>> +2. Registering/Unregistering the NVMEM provider
>> +===============================================
>> +
>> +A NVMEM provider can register with NVMEM core by suppling relevant
> ^
> supplying
>
>> +nvmem configuration to nvmem_register(), on success core would return a valid
>> +nvmem_device pointer.
>> +
>> +nvmem_unregister(nvmem) is used to unregister the already registered provider.
>
> unregister a previously registered provider?
>
>> +
>> +For example for simple qfprom case:
>
> For example, a simple qfprom case:
>
oops.. I will fix it.
>> +
>> +static struct nvmem_config econfig = {
>> + .name = "qfprom",
>> + .owner = THIS_MODULE,
>> +};
>> +
>> +static int qfprom_probe(struct platform_device *pdev)
>> +{
>> + ...
>> + econfig.dev = &pdev->dev;
>> + nvmem = nvmem_register(&econfig);
>> + ...
>> +}
>> +
>> +It is mandatory that the NVMEM provider has a regmap associated with its
>> +struct device.
>
> How do I ensure that?
yes, I think I need to add few lines on the errors which would make it
more explicit.
>
>> +
>> +NVMEM Consumers
>> ++++++++++++++++
>> +
>> +NVMEM consumers are the entities which make use of the NVMEM provider to
>> +read/write into NVMEM.
>
> read from and write to NVMEM?
>
Yep.
>> +
>> +3. NVMEM cell based consumer APIs.
>> +=================================
>> +
>> +NVMEM cells are the data entries/fields in the NVMEM.
>> +The NVMEM framework provides 3 APIs to read/write NVMEM cells.
>> +
>> +struct nvmem_cell *nvmem_cell_get(struct device *dev, const char *name);
>> +struct nvmem_cell *devm_nvmem_cell_get(struct device *dev, const char *name);
>> +
>> +void nvmem_cell_put(struct nvmem_cell *cell);
>> +void devm_nvmem_cell_put(struct device *dev, struct nvmem_cell *cell);
>> +
>> +void *nvmem_cell_read(struct nvmem_cell *cell, ssize_t *len);
>> +int nvmem_cell_write(struct nvmem_cell *cell, void *buf, ssize_t len);
>> +
>> +*nvmem_cell_get() apis will get a reference to nvmem cell for a given id,
>> +and nvmem_cell_read/write() can then directly read or write to the cell.
>
> Drop "directly"?
>
ok.
>> +Once the usage of the cell is finished the consumer should call *nvmem_cell_put()
>> +to free all the allocation memory for the cell.
>> +
>> +4. Direct NVMEM device based consumer APIs.
> ^
> Drop the full stop?
>
>> +==========================================
>> +
>> +In some instances it is necessary to directly read/write the NVMEM.
>> +To facilitate such consumers NVMEM framework provides below apis.
>> +
>> +struct nvmem_device *nvmem_device_get(struct device *dev, const char *name);
>> +struct nvmem_device *devm_nvmem_device_get(struct device *dev,
>> + const char *name);
>> +void nvmem_device_put(struct nvmem_device *nvmem);
>> +int nvmem_device_read(struct nvmem_device *nvmem, unsigned int offset,
>> + size_t bytes, void *buf);
>> +int nvmem_device_write(struct nvmem_device *nvmem, unsigned int offset,
>> + size_t bytes, void *buf);
>> +int nvmem_device_cell_read(struct nvmem_device *nvmem,
>> + struct nvmem_cell_info *info, void *buf);
>> +int nvmem_device_cell_write(struct nvmem_device *nvmem,
>> + struct nvmem_cell_info *info, void *buf);
>> +
>> +Before the consumers can read/write NVMEM directly, it should get hold
> ^
> a
>> +of nvmem_controller from one of the *nvmem_device_get() api.
>> +
>> +Difference between these apis and cell based apis is that these apis
> ^
> The
>
yes, will fix it.
>> +always take nvmem_device as parameter.
>> +
>> +5. Releasing a reference to the NVMEM
>> +=====================================
>> +
>> +When the consumers no longer needs the NVMEM, it has to release the reference
>
> When a consumer no longer needs?
yep I will fix it.
>
>> +to the NVMEM it has obtained using the APIs mentioned in the above section.
>> +NVMEM framework provides 2 APIs to release a reference to the NVMEM.
> ^
> The
>
>> +
>> +void nvmem_cell_put(struct nvmem_cell *cell);
>> +void devm_nvmem_cell_put(struct device *dev, struct nvmem_cell *cell);
>> +void nvmem_device_put(struct nvmem_device *nvmem);
>> +void devm_nvmem_device_put(struct device *dev, struct nvmem_device *nvmem);
>> +
>> +Both these APIs are used to release a reference to the NVMEM and
>> +devm_nvmem_cell_put and devm_nvmem_device_put destroys the devres associated
>> +with this NVMEM.
>
> s/this/the/
sure, will fix it.
>
>> +
>> +Userspace
>> ++++++++++
>> +
>> +6. Userspace binary interface.
> ^
> Drop the full stop?
>
>> +==============================
>> +
>> +Userspace can read/write the raw NVMEM file located at
>> +/sys/bus/nvmem/devices/*/nvmem
>> +
>> +ex:
>> +
>> +hexdump /sys/bus/nvmem/devices/qfprom0/nvmem
>> +
>> +0000000 0000 0000 0000 0000 0000 0000 0000 0000
>> +*
>> +00000a0 db10 2240 0000 e000 0c00 0c00 0000 0c00
>> +0000000 0000 0000 0000 0000 0000 0000 0000 0000
>> +...
>> +*
>> +0001000
>> +
>> +7. DeviceTree Binding
>> +=====================
>> +
>> +The documentation for NVMEM dt binding can be found @
>> +Documentation/devicetree/bindings/nvmem/nvmem.txt
>
> How about?
>
> See Documentation/devicetree/bindings/nvmem/nvmem.txt
sounds good.
>
^ permalink raw reply
* Re: [PATCH v7 5/9] Documentation: nvmem: add nvmem api level and how-to doc
From: Stephen Boyd @ 2015-07-14 21:32 UTC (permalink / raw)
To: Srinivas Kandagatla
Cc: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
Greg Kroah-Hartman, Rob Herring, Kumar Gala, Mark Brown,
s.hauer-bIcnvbaLZ9MEGnE8C9+IrQ, linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-arm-msm-u79uwXL29TY76Z2rM5mHXA, arnd-r2nGTMty4D4,
pantelis.antoniou-OWPKS81ov/FWk0Htik3J/w,
mporter-OWPKS81ov/FWk0Htik3J/w, stefan.wahren-eS4NqCHxEME,
wxt-TNX95d0MmH7DzftRWevZcw
In-Reply-To: <1436521521-10889-1-git-send-email-srinivas.kandagatla-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
On 07/10, Srinivas Kandagatla wrote:
> diff --git a/Documentation/nvmem/nvmem.txt b/Documentation/nvmem/nvmem.txt
> new file mode 100644
> index 0000000..b074b71
> --- /dev/null
> +++ b/Documentation/nvmem/nvmem.txt
> @@ -0,0 +1,152 @@
> + NVMEM SUBSYSTEM
> + Srinivas Kandagatla <srinivas.kandagatla-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
> +
> +This document explains the Simple NVMEM Framework along with the APIs provided,
Why is simple and framework capitalized? Is it the "Simple NVMEM
Framework" or just the "NVMEM" framework?
> +and how-to-use.
how to use it?
> +
> +1. Introduction
> +===============
> +*NVMEM* is the abbreviation for Non Volatile Memory layer. It is used to
> +retrieve configuration or SOC or Device specific data from a non volatile memories
^ ^
of remove a?
> +like eeprom, efuses and so on.
> +
> +Up until now, NVMEM drivers like eeprom were stored in drivers/misc, where they
Up until now will soon be out of date, perhaps say "before this
framework existed"?
> +all had to duplicate pretty much the same code to register a sysfs file, allow
> +in-kernel users to access the content of the devices they were driving, etc.
> +
> +This was also a problem as far as other in-kernel users were involved, since
> +the solutions used were pretty much different from on driver to another, there
^
one
> +was a rather big abstraction leak.
> +
> +Introduction of this framework aims at solving this. It also introduces DT
This framework aims to solve these problems.
> +representation for consumer devices to go get the data they require (MAC
> +Addresses, SoC/Revision ID, part numbers, and so on) from the NVMEMs.
> +This framework is based on regmap, so that most of the abstraction
> +available in regmap can be reused, across multiple types of buses.
> +
> +NVMEM Providers
> ++++++++++++++++
> +
> +NVMEM provider refers to an entity that implements methods to initialize, read
> +and write the non-volatile memory.
> +
> +2. Registering/Unregistering the NVMEM provider
> +===============================================
> +
> +A NVMEM provider can register with NVMEM core by suppling relevant
^
supplying
> +nvmem configuration to nvmem_register(), on success core would return a valid
> +nvmem_device pointer.
> +
> +nvmem_unregister(nvmem) is used to unregister the already registered provider.
unregister a previously registered provider?
> +
> +For example for simple qfprom case:
For example, a simple qfprom case:
> +
> +static struct nvmem_config econfig = {
> + .name = "qfprom",
> + .owner = THIS_MODULE,
> +};
> +
> +static int qfprom_probe(struct platform_device *pdev)
> +{
> + ...
> + econfig.dev = &pdev->dev;
> + nvmem = nvmem_register(&econfig);
> + ...
> +}
> +
> +It is mandatory that the NVMEM provider has a regmap associated with its
> +struct device.
How do I ensure that?
> +
> +NVMEM Consumers
> ++++++++++++++++
> +
> +NVMEM consumers are the entities which make use of the NVMEM provider to
> +read/write into NVMEM.
read from and write to NVMEM?
> +
> +3. NVMEM cell based consumer APIs.
> +=================================
> +
> +NVMEM cells are the data entries/fields in the NVMEM.
> +The NVMEM framework provides 3 APIs to read/write NVMEM cells.
> +
> +struct nvmem_cell *nvmem_cell_get(struct device *dev, const char *name);
> +struct nvmem_cell *devm_nvmem_cell_get(struct device *dev, const char *name);
> +
> +void nvmem_cell_put(struct nvmem_cell *cell);
> +void devm_nvmem_cell_put(struct device *dev, struct nvmem_cell *cell);
> +
> +void *nvmem_cell_read(struct nvmem_cell *cell, ssize_t *len);
> +int nvmem_cell_write(struct nvmem_cell *cell, void *buf, ssize_t len);
> +
> +*nvmem_cell_get() apis will get a reference to nvmem cell for a given id,
> +and nvmem_cell_read/write() can then directly read or write to the cell.
Drop "directly"?
> +Once the usage of the cell is finished the consumer should call *nvmem_cell_put()
> +to free all the allocation memory for the cell.
> +
> +4. Direct NVMEM device based consumer APIs.
^
Drop the full stop?
> +==========================================
> +
> +In some instances it is necessary to directly read/write the NVMEM.
> +To facilitate such consumers NVMEM framework provides below apis.
> +
> +struct nvmem_device *nvmem_device_get(struct device *dev, const char *name);
> +struct nvmem_device *devm_nvmem_device_get(struct device *dev,
> + const char *name);
> +void nvmem_device_put(struct nvmem_device *nvmem);
> +int nvmem_device_read(struct nvmem_device *nvmem, unsigned int offset,
> + size_t bytes, void *buf);
> +int nvmem_device_write(struct nvmem_device *nvmem, unsigned int offset,
> + size_t bytes, void *buf);
> +int nvmem_device_cell_read(struct nvmem_device *nvmem,
> + struct nvmem_cell_info *info, void *buf);
> +int nvmem_device_cell_write(struct nvmem_device *nvmem,
> + struct nvmem_cell_info *info, void *buf);
> +
> +Before the consumers can read/write NVMEM directly, it should get hold
^
a
> +of nvmem_controller from one of the *nvmem_device_get() api.
> +
> +Difference between these apis and cell based apis is that these apis
^
The
> +always take nvmem_device as parameter.
> +
> +5. Releasing a reference to the NVMEM
> +=====================================
> +
> +When the consumers no longer needs the NVMEM, it has to release the reference
When a consumer no longer needs?
> +to the NVMEM it has obtained using the APIs mentioned in the above section.
> +NVMEM framework provides 2 APIs to release a reference to the NVMEM.
^
The
> +
> +void nvmem_cell_put(struct nvmem_cell *cell);
> +void devm_nvmem_cell_put(struct device *dev, struct nvmem_cell *cell);
> +void nvmem_device_put(struct nvmem_device *nvmem);
> +void devm_nvmem_device_put(struct device *dev, struct nvmem_device *nvmem);
> +
> +Both these APIs are used to release a reference to the NVMEM and
> +devm_nvmem_cell_put and devm_nvmem_device_put destroys the devres associated
> +with this NVMEM.
s/this/the/
> +
> +Userspace
> ++++++++++
> +
> +6. Userspace binary interface.
^
Drop the full stop?
> +==============================
> +
> +Userspace can read/write the raw NVMEM file located at
> +/sys/bus/nvmem/devices/*/nvmem
> +
> +ex:
> +
> +hexdump /sys/bus/nvmem/devices/qfprom0/nvmem
> +
> +0000000 0000 0000 0000 0000 0000 0000 0000 0000
> +*
> +00000a0 db10 2240 0000 e000 0c00 0c00 0000 0c00
> +0000000 0000 0000 0000 0000 0000 0000 0000 0000
> +...
> +*
> +0001000
> +
> +7. DeviceTree Binding
> +=====================
> +
> +The documentation for NVMEM dt binding can be found @
> +Documentation/devicetree/bindings/nvmem/nvmem.txt
How about?
See Documentation/devicetree/bindings/nvmem/nvmem.txt
--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
^ permalink raw reply
* Re: [PATCH v7 6/9] nvmem: qfprom: Add Qualcomm QFPROM support.
From: Stephen Boyd @ 2015-07-14 21:18 UTC (permalink / raw)
To: Srinivas Kandagatla
Cc: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
Greg Kroah-Hartman, Rob Herring, Kumar Gala, Mark Brown,
s.hauer-bIcnvbaLZ9MEGnE8C9+IrQ, linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-arm-msm-u79uwXL29TY76Z2rM5mHXA, arnd-r2nGTMty4D4,
pantelis.antoniou-OWPKS81ov/FWk0Htik3J/w,
mporter-OWPKS81ov/FWk0Htik3J/w, stefan.wahren-eS4NqCHxEME,
wxt-TNX95d0MmH7DzftRWevZcw
In-Reply-To: <1436521529-10931-1-git-send-email-srinivas.kandagatla-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
On 07/10, Srinivas Kandagatla wrote:
> This patch adds QFPROM support driver which is used by other drivers
> like thermal sensor and cpufreq.
>
> On MSM parts there are some efuses (called qfprom) these fuses store
> things like calibration data, speed bins.. etc. Drivers like cpufreq,
> thermal sensors would read out this data for configuring the driver.
>
> Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
> ---
I didn't see any reply on v6, but at least delay.h was removed.
Same comments from v6 below.
> diff --git a/drivers/nvmem/qfprom.c b/drivers/nvmem/qfprom.c
> new file mode 100644
> index 0000000..0e22a03
> --- /dev/null
> +++ b/drivers/nvmem/qfprom.c
> @@ -0,0 +1,87 @@
> +/*
> + * Copyright (C) 2015 Srinivas Kandagatla <srinivas.kandagatla-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <linux/device.h>
> +#include <linux/io.h>
Is this include used?
> +#include <linux/module.h>
> +#include <linux/nvmem-provider.h>
> +#include <linux/platform_device.h>
> +#include <linux/regmap.h>
> +#include <linux/slab.h>
> +
> +static struct regmap_config qfprom_regmap_config = {
Can this be const?
> + .reg_bits = 32,
> + .val_bits = 8,
> + .reg_stride = 1,
> +};
> +
> +static struct nvmem_config econfig = {
Can this be const?
> + .name = "qfprom",
> + .owner = THIS_MODULE,
> +};
> +
--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
^ permalink raw reply
* Re: [PATCH -mm v7 5/6] proc: add kpageidle file
From: Andres Lagar-Cavilla @ 2015-07-14 20:27 UTC (permalink / raw)
To: Vladimir Davydov
Cc: Andrew Morton, Minchan Kim, Raghavendra K T, Johannes Weiner,
Michal Hocko, Greg Thelen, Michel Lespinasse, David Rientjes,
Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-doc-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
cgroups-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20150714110527.GA1015@esperanza>
On Tue, Jul 14, 2015 at 4:05 AM, Vladimir Davydov
<vdavydov-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org> wrote:
> On Mon, Jul 13, 2015 at 12:02:57PM -0700, Andres Lagar-Cavilla wrote:
>> On Sat, Jul 11, 2015 at 7:48 AM, Vladimir Davydov
>> <vdavydov-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org> wrote:
> [...]
>> > +static struct page *kpageidle_get_page(unsigned long pfn)
>> > +{
>> > + struct page *page;
>> > + struct zone *zone;
>> > +
>> > + if (!pfn_valid(pfn))
>> > + return NULL;
>> > +
>> > + page = pfn_to_page(pfn);
>> > + if (!page || PageTail(page) || !PageLRU(page) ||
>> > + !get_page_unless_zero(page))
>>
>> get_page_unless_zero does not succeed for Tail pages.
>
> True. So we don't seem to need the PageTail checks here at all, because
> if kpageidle_get_page succeeds, the page must be a head, so that we
> won't dive into expensive rmap_walk for tail pages. Will remove it then.
>
>>
>> > + return NULL;
>> > +
>> > + if (unlikely(PageTail(page))) {
>> > + put_page(page);
>> > + return NULL;
>> > + }
>> > +
>> > + zone = page_zone(page);
>> > + spin_lock_irq(&zone->lru_lock);
>> > + if (unlikely(!PageLRU(page))) {
>> > + put_page(page);
>> > + page = NULL;
>> > + }
>> > + spin_unlock_irq(&zone->lru_lock);
>> > + return page;
>> > +}
>> > +
>> > +static int kpageidle_clear_pte_refs_one(struct page *page,
>> > + struct vm_area_struct *vma,
>> > + unsigned long addr, void *arg)
>> > +{
>> > + struct mm_struct *mm = vma->vm_mm;
>> > + spinlock_t *ptl;
>> > + pmd_t *pmd;
>> > + pte_t *pte;
>> > + bool referenced = false;
>> > +
>> > + if (unlikely(PageTransHuge(page))) {
>>
>> VM_BUG_ON(!PageHead)?
>
> Don't think it's necessary, because PageTransHuge already does this sort
> of check:
>
> : static inline int PageTransHuge(struct page *page)
> : {
> : VM_BUG_ON_PAGE(PageTail(page), page);
> : return PageHead(page);
> : }
>
>>
>> > + pmd = page_check_address_pmd(page, mm, addr,
>> > + PAGE_CHECK_ADDRESS_PMD_FLAG, &ptl);
>> > + if (pmd) {
>> > + referenced = pmdp_test_and_clear_young(vma, addr, pmd);
>>
>> For any workload using MMU notifiers, this will lose significant
>> information by not querying the secondary PTE. The most
>> straightforward case is KVM. Once mappings are setup, all access
>> activity is recorded through shadow PTEs. This interface will say
>> "idle" even though the VM is blasting memory.
>
> Hmm, interesting. It seems we have to introduce
> mmu_notifier_ops.clear_young then, which, in contrast to
> clear_flush_young, won't flush TLB. Looking back at your comment to v6,
> now I see that you already mentioned it, but I missed your point :-(
> OK, will do it in the next iteration.
There's clearly value in fixing things for KVM, but I don't have
knowledge of the other MMU notifiers. I like clear_young, maybe other
mmu notifiers will turn this into a no-op().
mmmmhh. What about TLB flushing in the mmu notifier? I guess that can
be internal to each implementation.
Andres
>
> Thanks a lot for the review!
>
> Vladimir
--
Andres Lagar-Cavilla | Google Kernel Team | andreslc-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org
^ permalink raw reply
* Re: [PATCHSET v4 0/5] pagemap: make useable for non-privilege users
From: Konstantin Khlebnikov @ 2015-07-14 20:15 UTC (permalink / raw)
To: Andrew Morton
Cc: Konstantin Khlebnikov,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, Naoya Horiguchi,
Kirill A. Shutemov, Mark Williamson, Linux Kernel Mailing List,
Linux API
In-Reply-To: <20150714115252.8f21cfa864935a4b403c3d8d-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>
On Tue, Jul 14, 2015 at 9:52 PM, Andrew Morton
<akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org> wrote:
> On Tue, 14 Jul 2015 18:37:34 +0300 Konstantin Khlebnikov <khlebnikov-XoJtRXgx1JseBXzfvpsJ4g@public.gmane.org> wrote:
>
>> This patchset makes pagemap useable again in the safe way (after row hammer
>> bug it was made CAP_SYS_ADMIN-only). This patchset restores access for
>> non-privileged users but hides PFNs from them.
>
> Documentation/vm/pagemap.txt hasn't been updated to describe these
> privilege issues?
Will do. Too much time passed between versions, I planned but forgot about that.
>
>> Also it adds bit 'map-exlusive' which is set if page is mapped only here:
>> it helps in estimation of working set without exposing pfns and allows to
>> distinguish CoWed and non-CoWed private anonymous pages.
>>
>> Second patch removes page-shift bits and completes migration to the new
>> pagemap format: flags soft-dirty and mmap-exlusive are available only
>> in the new format.
>
> I'm not really seeing a description of the new format in these
> changelogs. Precisely what got removed, what got added and which
> capabilities change the output in what manner?
Now pfn (bits 0-54) is zero if task who opened pagemap has no
CAP_SYS_ADMIN (system-wide).
in v2 format page-shift (bits 55-60) now used for flags:
55 - soft-dirty (added for checkpoint-restore, I guess)
56 - mmap-exclusive (added in last patch)
57-60 - free for use
I'll document the history of these changes.
>
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo-Bw31MaZKKs0EbZ0PF+XxCw@public.gmane.org For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org"> email-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org </a>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox