* Re: [PATCH v2 6/7] mm, proc: add KReclaimable to /proc/meminfo
From: Minchan Kim @ 2018-06-19 8:13 UTC (permalink / raw)
To: Vlastimil Babka
Cc: Andrew Morton, linux-mm, linux-kernel, Roman Gushchin,
Michal Hocko, Johannes Weiner, linux-api, Christoph Lameter,
David Rientjes, Mel Gorman, Matthew Wilcox
In-Reply-To: <650c3fab-3137-4fe6-272a-f4ec104855a7@suse.cz>
On Tue, Jun 19, 2018 at 09:30:03AM +0200, Vlastimil Babka wrote:
> On 06/18/2018 11:33 PM, Andrew Morton wrote:
> > On Mon, 18 Jun 2018 11:18:07 +0200 Vlastimil Babka <vbabka@suse.cz> wrote:
> >
> >> The vmstat NR_KERNEL_MISC_RECLAIMABLE counter is for kernel non-slab
> >> allocations that can be reclaimed via shrinker. In /proc/meminfo, we can show
> >> the sum of all reclaimable kernel allocations (including slab) as
> >> "KReclaimable". Add the same counter also to per-node meminfo under /sys
> >
> > Why do you consider this useful enough to justify adding it to
> > /pro/meminfo? How will people use it, what benefit will they see, etc?
>
> Let's add this:
>
> With this counter, users will have more complete information about
> kernel memory usage. Non-slab reclaimable pages (currently just the ION
> allocator) will not be missing from /proc/meminfo, making users wonder
> where part of their memory went. More precisely, they already appear in
> MemAvailable, but without the new counter, it's not obvious why the
> value in MemAvailable doesn't fully correspond with the sum of other
> counters participating in it.
Hmm, if we could get MemAvailable with sum of other counters participating
in it, MemAvailable wouldn't be meaninful. IMO, MemAvailable don't need to
be matched with other counters.
The benefit of ION KReclaimable in real field is there are some sluggish
problem bugreport under memory pressure and found ION page pool is too
much without shrinking. In that case, that meminfo would be useful to
know something was broken in the system.
In that point of view, a concern to me is if we put more KReclaimable
pages(e.g., binder is candidate), it ends up we couldn't identify what
caches are too much among them. That means we needs KReclaimableInfo(like
slabinfo) to show each type's KReclaimable pages in future.
Anyway, it's good for first step.
>
> > Maybe you've undersold this whole patchset, but I'm struggling a bit to
> > see what the end-user benefits are. What would be wrong with just
> > sticking with what we have now?
>
> Fair enough, I will add more info in reply to the cover letter.
>
^ permalink raw reply
* Re: [PATCH v2 0/7] kmalloc-reclaimable caches
From: Vlastimil Babka @ 2018-06-19 7:54 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-kernel, Roman Gushchin, Michal Hocko,
Johannes Weiner, linux-api, Christoph Lameter, David Rientjes,
Mel Gorman, Matthew Wilcox, Laura Abbott, Sumit Semwal,
Vijayanand Jitta
In-Reply-To: <20180618091808.4419-1-vbabka@suse.cz>
On 06/18/2018 11:18 AM, Vlastimil Babka wrote:
> v2 changes:
> - shorten cache names to kmalloc-rcl-<SIZE>
> - last patch shortens <SIZE> for all kmalloc caches to e.g. "1k", "4M"
> - include dma caches to the 2D kmalloc_caches[] array to avoid a branch
> - vmstat counter nr_indirectly_reclaimable_bytes renamed to
> nr_kernel_misc_reclaimable, doesn't include kmalloc-rcl-*
> - /proc/meminfo counter renamed to KReclaimable, includes kmalloc-rcl*
> and nr_kernel_misc_reclaimable
>
> Hi,
>
> as discussed at LSF/MM [1] here's a patchset that introduces
> kmalloc-reclaimable caches (more details in the second patch) and uses them for
> SLAB freelists and dcache external names. The latter allows us to repurpose the
> NR_INDIRECTLY_RECLAIMABLE_BYTES counter later in the series.
More info about user benefits of the patchset:
With patch 4, dcache external names are allocated from kmalloc-rcl-*
caches, eliminating the need for manual accounting. More importantly, it
also ensures the reclaimable kmalloc allocations are grouped in pages
separate from the regular kmalloc allocations. The need for proper
accounting of dcache external names has shown it's easy for misbehaving
process to allocate lots of them, causing premature OOMs. Without the
added grouping, it's likely that similar workload can interleave the
dcache external names allocations with regular kmalloc allocations
(note: I haven't searched myself for an example of such regular kmalloc
allocation, but I would be very surprised if there wasn't some). A
pathological case would be e.g. one 64byte regular allocations with 63
external dcache names in a page (64x64=4096), which means the page is
not freed even after reclaiming after all dcache names, and the process
can thus steal the whole page with single 64byte allocation.
If there other kmalloc users similar to dcache external names become
identified, they can also benefit from the new functionality simply by
adding __GFP_RECLAIMABLE to the kmalloc calls.
Side benefits of the patchset (that could be also merged separately)
include removed branch for detecting __GFP_DMA kmalloc(), and shortening
kmalloc cache names in /proc/slabinfo output. The latter is potentially
an ABI break in case there are tools parsing the names and expecting the
values to be in bytes.
^ permalink raw reply
* Re: [PATCH v2 6/7] mm, proc: add KReclaimable to /proc/meminfo
From: Vlastimil Babka @ 2018-06-19 7:30 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-kernel, Roman Gushchin, Michal Hocko,
Johannes Weiner, linux-api, Christoph Lameter, David Rientjes,
Mel Gorman, Matthew Wilcox
In-Reply-To: <20180618143317.eb8f5d7b6c667784343ef902@linux-foundation.org>
On 06/18/2018 11:33 PM, Andrew Morton wrote:
> On Mon, 18 Jun 2018 11:18:07 +0200 Vlastimil Babka <vbabka@suse.cz> wrote:
>
>> The vmstat NR_KERNEL_MISC_RECLAIMABLE counter is for kernel non-slab
>> allocations that can be reclaimed via shrinker. In /proc/meminfo, we can show
>> the sum of all reclaimable kernel allocations (including slab) as
>> "KReclaimable". Add the same counter also to per-node meminfo under /sys
>
> Why do you consider this useful enough to justify adding it to
> /pro/meminfo? How will people use it, what benefit will they see, etc?
Let's add this:
With this counter, users will have more complete information about
kernel memory usage. Non-slab reclaimable pages (currently just the ION
allocator) will not be missing from /proc/meminfo, making users wonder
where part of their memory went. More precisely, they already appear in
MemAvailable, but without the new counter, it's not obvious why the
value in MemAvailable doesn't fully correspond with the sum of other
counters participating in it.
> Maybe you've undersold this whole patchset, but I'm struggling a bit to
> see what the end-user benefits are. What would be wrong with just
> sticking with what we have now?
Fair enough, I will add more info in reply to the cover letter.
^ permalink raw reply
* Re: [PATCH v2 6/7] mm, proc: add KReclaimable to /proc/meminfo
From: Andrew Morton @ 2018-06-18 21:33 UTC (permalink / raw)
To: Vlastimil Babka
Cc: linux-mm, linux-kernel, Roman Gushchin, Michal Hocko,
Johannes Weiner, linux-api, Christoph Lameter, David Rientjes,
Mel Gorman, Matthew Wilcox
In-Reply-To: <20180618091808.4419-7-vbabka@suse.cz>
On Mon, 18 Jun 2018 11:18:07 +0200 Vlastimil Babka <vbabka@suse.cz> wrote:
> The vmstat NR_KERNEL_MISC_RECLAIMABLE counter is for kernel non-slab
> allocations that can be reclaimed via shrinker. In /proc/meminfo, we can show
> the sum of all reclaimable kernel allocations (including slab) as
> "KReclaimable". Add the same counter also to per-node meminfo under /sys
Why do you consider this useful enough to justify adding it to
/pro/meminfo? How will people use it, what benefit will they see, etc?
Maybe you've undersold this whole patchset, but I'm struggling a bit to
see what the end-user benefits are. What would be wrong with just
sticking with what we have now?
^ permalink raw reply
* [PATCH v2 7/7] mm, slab: shorten kmalloc cache names for large sizes
From: Vlastimil Babka @ 2018-06-18 9:18 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-kernel, Roman Gushchin, Michal Hocko,
Johannes Weiner, linux-api, Christoph Lameter, David Rientjes,
Mel Gorman, Matthew Wilcox, Vlastimil Babka
In-Reply-To: <20180618091808.4419-1-vbabka@suse.cz>
Kmalloc cache names can get quite long for large object sizes, when the sizes
are expressed in bytes. Use 'k' and 'M' prefixes to make the names as short
as possible e.g. in /proc/slabinfo. This works, as we mostly use power-of-two
sizes, with exceptions only below 1k.
Example: 'kmalloc-4194304' becomes 'kmalloc-4M'
Suggested-by: Matthew Wilcox <willy@infradead.org>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
mm/slab_common.c | 38 ++++++++++++++++++++++++++------------
1 file changed, 26 insertions(+), 12 deletions(-)
diff --git a/mm/slab_common.c b/mm/slab_common.c
index 8a30d6979936..462509978ec0 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -1045,15 +1045,15 @@ const struct kmalloc_info_struct kmalloc_info[] __initconst = {
{"kmalloc-16", 16}, {"kmalloc-32", 32},
{"kmalloc-64", 64}, {"kmalloc-128", 128},
{"kmalloc-256", 256}, {"kmalloc-512", 512},
- {"kmalloc-1024", 1024}, {"kmalloc-2048", 2048},
- {"kmalloc-4096", 4096}, {"kmalloc-8192", 8192},
- {"kmalloc-16384", 16384}, {"kmalloc-32768", 32768},
- {"kmalloc-65536", 65536}, {"kmalloc-131072", 131072},
- {"kmalloc-262144", 262144}, {"kmalloc-524288", 524288},
- {"kmalloc-1048576", 1048576}, {"kmalloc-2097152", 2097152},
- {"kmalloc-4194304", 4194304}, {"kmalloc-8388608", 8388608},
- {"kmalloc-16777216", 16777216}, {"kmalloc-33554432", 33554432},
- {"kmalloc-67108864", 67108864}
+ {"kmalloc-1k", 1024}, {"kmalloc-2k", 2048},
+ {"kmalloc-4k", 4096}, {"kmalloc-8k", 8192},
+ {"kmalloc-16k", 16384}, {"kmalloc-32k", 32768},
+ {"kmalloc-64k", 65536}, {"kmalloc-128k", 131072},
+ {"kmalloc-256k", 262144}, {"kmalloc-512k", 524288},
+ {"kmalloc-1M", 1048576}, {"kmalloc-2M", 2097152},
+ {"kmalloc-4M", 4194304}, {"kmalloc-8M", 8388608},
+ {"kmalloc-16M", 16777216}, {"kmalloc-32M", 33554432},
+ {"kmalloc-64M", 67108864}
};
/*
@@ -1103,6 +1103,21 @@ void __init setup_kmalloc_cache_index_table(void)
}
}
+static const char *
+kmalloc_cache_name(const char *prefix, unsigned int size)
+{
+
+ static const char units[3] = "\0kM";
+ int idx = 0;
+
+ while (size >= 1024 && (size % 1024 == 0)) {
+ size /= 1024;
+ idx++;
+ }
+
+ return kasprintf(GFP_NOWAIT, "%s-%u%c", prefix, size, units[idx]);
+}
+
static void __init
new_kmalloc_cache(int idx, int type, slab_flags_t flags)
{
@@ -1110,7 +1125,7 @@ new_kmalloc_cache(int idx, int type, slab_flags_t flags)
if (type == KMALLOC_RECLAIM) {
flags |= SLAB_RECLAIM_ACCOUNT;
- name = kasprintf(GFP_NOWAIT, "kmalloc-rcl-%u",
+ name = kmalloc_cache_name("kmalloc-rcl",
kmalloc_info[idx].size);
BUG_ON(!name);
} else {
@@ -1159,8 +1174,7 @@ void __init create_kmalloc_caches(slab_flags_t flags)
if (s) {
unsigned int size = kmalloc_size(i);
- char *n = kasprintf(GFP_NOWAIT,
- "dma-kmalloc-%u", size);
+ const char *n = kmalloc_cache_name("dma-kmalloc", size);
BUG_ON(!n);
kmalloc_caches[KMALLOC_DMA][i] = create_kmalloc_cache(
--
2.17.1
^ permalink raw reply related
* [PATCH v2 6/7] mm, proc: add KReclaimable to /proc/meminfo
From: Vlastimil Babka @ 2018-06-18 9:18 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-kernel, Roman Gushchin, Michal Hocko,
Johannes Weiner, linux-api, Christoph Lameter, David Rientjes,
Mel Gorman, Matthew Wilcox, Vlastimil Babka
In-Reply-To: <20180618091808.4419-1-vbabka@suse.cz>
The vmstat NR_KERNEL_MISC_RECLAIMABLE counter is for kernel non-slab
allocations that can be reclaimed via shrinker. In /proc/meminfo, we can show
the sum of all reclaimable kernel allocations (including slab) as
"KReclaimable". Add the same counter also to per-node meminfo under /sys
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
Documentation/filesystems/proc.txt | 4 ++++
drivers/base/node.c | 19 ++++++++++++-------
fs/proc/meminfo.c | 16 ++++++++--------
3 files changed, 24 insertions(+), 15 deletions(-)
diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt
index 520f6a84cf50..6a255f960ab5 100644
--- a/Documentation/filesystems/proc.txt
+++ b/Documentation/filesystems/proc.txt
@@ -858,6 +858,7 @@ Writeback: 0 kB
AnonPages: 861800 kB
Mapped: 280372 kB
Shmem: 644 kB
+KReclaimable: 168048 kB
Slab: 284364 kB
SReclaimable: 159856 kB
SUnreclaim: 124508 kB
@@ -921,6 +922,9 @@ AnonHugePages: Non-file backed huge pages mapped into userspace page tables
ShmemHugePages: Memory used by shared memory (shmem) and tmpfs allocated
with huge pages
ShmemPmdMapped: Shared memory mapped into userspace with huge pages
+KReclaimable: Kernel allocations that the kernel will attempt to reclaim
+ under memory pressure. Includes SReclaimable (below), and other
+ direct allocations with a shrinker.
Slab: in-kernel data structures cache
SReclaimable: Part of Slab, that might be reclaimed, such as caches
SUnreclaim: Part of Slab, that cannot be reclaimed on memory pressure
diff --git a/drivers/base/node.c b/drivers/base/node.c
index a5e821d09656..81cef8031eae 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -67,8 +67,11 @@ static ssize_t node_read_meminfo(struct device *dev,
int nid = dev->id;
struct pglist_data *pgdat = NODE_DATA(nid);
struct sysinfo i;
+ unsigned long sreclaimable, sunreclaimable;
si_meminfo_node(&i, nid);
+ sreclaimable = node_page_state(pgdat, NR_SLAB_RECLAIMABLE);
+ sunreclaimable = node_page_state(pgdat, NR_SLAB_UNRECLAIMABLE);
n = sprintf(buf,
"Node %d MemTotal: %8lu kB\n"
"Node %d MemFree: %8lu kB\n"
@@ -118,6 +121,7 @@ static ssize_t node_read_meminfo(struct device *dev,
"Node %d NFS_Unstable: %8lu kB\n"
"Node %d Bounce: %8lu kB\n"
"Node %d WritebackTmp: %8lu kB\n"
+ "Node %d KReclaimable: %8lu kB\n"
"Node %d Slab: %8lu kB\n"
"Node %d SReclaimable: %8lu kB\n"
"Node %d SUnreclaim: %8lu kB\n"
@@ -138,20 +142,21 @@ static ssize_t node_read_meminfo(struct device *dev,
nid, K(node_page_state(pgdat, NR_UNSTABLE_NFS)),
nid, K(sum_zone_node_page_state(nid, NR_BOUNCE)),
nid, K(node_page_state(pgdat, NR_WRITEBACK_TEMP)),
- nid, K(node_page_state(pgdat, NR_SLAB_RECLAIMABLE) +
- node_page_state(pgdat, NR_SLAB_UNRECLAIMABLE)),
- nid, K(node_page_state(pgdat, NR_SLAB_RECLAIMABLE)),
+ nid, K(sreclaimable +
+ node_page_state(pgdat, NR_KERNEL_MISC_RECLAIMABLE)),
+ nid, K(sreclaimable + sunreclaimable),
+ nid, K(sreclaimable),
+ nid, K(sunreclaimable)
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
- nid, K(node_page_state(pgdat, NR_SLAB_UNRECLAIMABLE)),
+ ,
nid, K(node_page_state(pgdat, NR_ANON_THPS) *
HPAGE_PMD_NR),
nid, K(node_page_state(pgdat, NR_SHMEM_THPS) *
HPAGE_PMD_NR),
nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) *
- HPAGE_PMD_NR));
-#else
- nid, K(node_page_state(pgdat, NR_SLAB_UNRECLAIMABLE)));
+ HPAGE_PMD_NR)
#endif
+ );
n += hugetlb_report_node_meminfo(nid, buf + n);
return n;
}
diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c
index 2fb04846ed11..61a18477bc07 100644
--- a/fs/proc/meminfo.c
+++ b/fs/proc/meminfo.c
@@ -37,6 +37,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
long cached;
long available;
unsigned long pages[NR_LRU_LISTS];
+ unsigned long sreclaimable, sunreclaim;
int lru;
si_meminfo(&i);
@@ -52,6 +53,8 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
pages[lru] = global_node_page_state(NR_LRU_BASE + lru);
available = si_mem_available();
+ sreclaimable = global_node_page_state(NR_SLAB_RECLAIMABLE);
+ sunreclaim = global_node_page_state(NR_SLAB_UNRECLAIMABLE);
show_val_kb(m, "MemTotal: ", i.totalram);
show_val_kb(m, "MemFree: ", i.freeram);
@@ -93,14 +96,11 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
show_val_kb(m, "Mapped: ",
global_node_page_state(NR_FILE_MAPPED));
show_val_kb(m, "Shmem: ", i.sharedram);
- show_val_kb(m, "Slab: ",
- global_node_page_state(NR_SLAB_RECLAIMABLE) +
- global_node_page_state(NR_SLAB_UNRECLAIMABLE));
-
- show_val_kb(m, "SReclaimable: ",
- global_node_page_state(NR_SLAB_RECLAIMABLE));
- show_val_kb(m, "SUnreclaim: ",
- global_node_page_state(NR_SLAB_UNRECLAIMABLE));
+ show_val_kb(m, "KReclaimable: ", sreclaimable +
+ global_node_page_state(NR_KERNEL_MISC_RECLAIMABLE));
+ show_val_kb(m, "Slab: ", sreclaimable + sunreclaim);
+ show_val_kb(m, "SReclaimable: ", sreclaimable);
+ show_val_kb(m, "SUnreclaim: ", sunreclaim);
seq_printf(m, "KernelStack: %8lu kB\n",
global_zone_page_state(NR_KERNEL_STACK_KB));
show_val_kb(m, "PageTables: ",
--
2.17.1
^ permalink raw reply related
* [PATCH v2 5/7] mm: rename and change semantics of nr_indirectly_reclaimable_bytes
From: Vlastimil Babka @ 2018-06-18 9:18 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-kernel, Roman Gushchin, Michal Hocko,
Johannes Weiner, linux-api, Christoph Lameter, David Rientjes,
Mel Gorman, Matthew Wilcox, Vlastimil Babka, Vijayanand Jitta,
Laura Abbott, Sumit Semwal
In-Reply-To: <20180618091808.4419-1-vbabka@suse.cz>
The vmstat counter NR_INDIRECTLY_RECLAIMABLE_BYTES was introduced by commit
eb59254608bc ("mm: introduce NR_INDIRECTLY_RECLAIMABLE_BYTES") with the goal of
accounting objects that can be reclaimed, but cannot be allocated via a
SLAB_RECLAIM_ACCOUNT cache. This is now possible via kmalloc() with
__GFP_RECLAIMABLE flag, and the dcache external names user is converted.
The counter is however still useful for accounting direct page allocations
(i.e. not slab) with a shrinker, such as the ION page pool. So keep it, and:
- change granularity to pages to be more like other counters; sub-page
allocations should be able to use kmalloc
- rename the counter to NR_KERNEL_MISC_RECLAIMABLE
- expose the counter again in vmstat as "nr_kernel_misc_reclaimable"; we can
again remove the check for not printing "hidden" counters
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Vijayanand Jitta <vjitta@codeaurora.org>
Cc: Laura Abbott <labbott@redhat.com>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
---
drivers/staging/android/ion/ion_page_pool.c | 4 ++--
include/linux/mmzone.h | 2 +-
mm/page_alloc.c | 19 +++++++------------
mm/util.c | 3 +--
mm/vmstat.c | 6 +-----
5 files changed, 12 insertions(+), 22 deletions(-)
diff --git a/drivers/staging/android/ion/ion_page_pool.c b/drivers/staging/android/ion/ion_page_pool.c
index 9bc56eb48d2a..b7ad2d2449ac 100644
--- a/drivers/staging/android/ion/ion_page_pool.c
+++ b/drivers/staging/android/ion/ion_page_pool.c
@@ -33,8 +33,8 @@ static void ion_page_pool_add(struct ion_page_pool *pool, struct page *page)
pool->low_count++;
}
- mod_node_page_state(page_pgdat(page), NR_INDIRECTLY_RECLAIMABLE_BYTES,
- (1 << (PAGE_SHIFT + pool->order)));
+ mod_node_page_state(page_pgdat(page), NR_KERNEL_MISC_RECLAIMABLE,
+ 1 << pool->order);
mutex_unlock(&pool->mutex);
}
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 32699b2dc52a..c2f6bc4c9e8a 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -180,7 +180,7 @@ enum node_stat_item {
NR_VMSCAN_IMMEDIATE, /* Prioritise for reclaim when writeback ends */
NR_DIRTIED, /* page dirtyings since bootup */
NR_WRITTEN, /* page writings since bootup */
- NR_INDIRECTLY_RECLAIMABLE_BYTES, /* measured in bytes */
+ NR_KERNEL_MISC_RECLAIMABLE, /* reclaimable non-slab kernel pages */
NR_VM_NODE_STAT_ITEMS
};
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 1521100f1e63..8ceb45e11b97 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -4704,6 +4704,7 @@ long si_mem_available(void)
unsigned long pagecache;
unsigned long wmark_low = 0;
unsigned long pages[NR_LRU_LISTS];
+ unsigned long reclaimable;
struct zone *zone;
int lru;
@@ -4729,19 +4730,13 @@ long si_mem_available(void)
available += pagecache;
/*
- * Part of the reclaimable slab consists of items that are in use,
- * and cannot be freed. Cap this estimate at the low watermark.
+ * Part of the reclaimable slab and other kernel memory consists of
+ * items that are in use, and cannot be freed. Cap this estimate at the
+ * low watermark.
*/
- available += global_node_page_state(NR_SLAB_RECLAIMABLE) -
- min(global_node_page_state(NR_SLAB_RECLAIMABLE) / 2,
- wmark_low);
-
- /*
- * Part of the kernel memory, which can be released under memory
- * pressure.
- */
- available += global_node_page_state(NR_INDIRECTLY_RECLAIMABLE_BYTES) >>
- PAGE_SHIFT;
+ reclaimable = global_node_page_state(NR_SLAB_RECLAIMABLE) +
+ global_node_page_state(NR_KERNEL_MISC_RECLAIMABLE);
+ available += reclaimable - min(reclaimable / 2, wmark_low);
if (available < 0)
available = 0;
diff --git a/mm/util.c b/mm/util.c
index 3351659200e6..891f0654e7b5 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -675,8 +675,7 @@ int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
* Part of the kernel memory, which can be released
* under memory pressure.
*/
- free += global_node_page_state(
- NR_INDIRECTLY_RECLAIMABLE_BYTES) >> PAGE_SHIFT;
+ free += global_node_page_state(NR_KERNEL_MISC_RECLAIMABLE);
/*
* Leave reserved pages. The pages are not for anonymous pages.
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 75eda9c2b260..7c677d3a61ec 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -1161,7 +1161,7 @@ const char * const vmstat_text[] = {
"nr_vmscan_immediate_reclaim",
"nr_dirtied",
"nr_written",
- "", /* nr_indirectly_reclaimable */
+ "nr_kernel_misc_reclaimable",
/* enum writeback_stat_item counters */
"nr_dirty_threshold",
@@ -1704,10 +1704,6 @@ static int vmstat_show(struct seq_file *m, void *arg)
unsigned long *l = arg;
unsigned long off = l - (unsigned long *)m->private;
- /* Skip hidden vmstat items. */
- if (*vmstat_text[off] == '\0')
- return 0;
-
seq_puts(m, vmstat_text[off]);
seq_put_decimal_ull(m, " ", *l);
seq_putc(m, '\n');
--
2.17.1
^ permalink raw reply related
* [PATCH v2 4/7] dcache: allocate external names from reclaimable kmalloc caches
From: Vlastimil Babka @ 2018-06-18 9:18 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-kernel, Roman Gushchin, Michal Hocko,
Johannes Weiner, linux-api, Christoph Lameter, David Rientjes,
Mel Gorman, Matthew Wilcox, Vlastimil Babka
In-Reply-To: <20180618091808.4419-1-vbabka@suse.cz>
We can use the newly introduced kmalloc-reclaimable-X caches, to allocate
external names in dcache, which will take care of the proper accounting
automatically, and also improve anti-fragmentation page grouping.
This effectively reverts commit f1782c9bc547 ("dcache: account external names
as indirectly reclaimable memory") and instead passes __GFP_RECLAIMABLE to
kmalloc(). The accounting thus moves from NR_INDIRECTLY_RECLAIMABLE_BYTES to
NR_SLAB_RECLAIMABLE, which is also considered in MemAvailable calculation and
overcommit decisions.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
fs/dcache.c | 38 +++++++++-----------------------------
1 file changed, 9 insertions(+), 29 deletions(-)
diff --git a/fs/dcache.c b/fs/dcache.c
index 0e8e5de3c48a..518c9ed8db8c 100644
--- a/fs/dcache.c
+++ b/fs/dcache.c
@@ -257,24 +257,10 @@ static void __d_free(struct rcu_head *head)
kmem_cache_free(dentry_cache, dentry);
}
-static void __d_free_external_name(struct rcu_head *head)
-{
- struct external_name *name = container_of(head, struct external_name,
- u.head);
-
- mod_node_page_state(page_pgdat(virt_to_page(name)),
- NR_INDIRECTLY_RECLAIMABLE_BYTES,
- -ksize(name));
-
- kfree(name);
-}
-
static void __d_free_external(struct rcu_head *head)
{
struct dentry *dentry = container_of(head, struct dentry, d_u.d_rcu);
-
- __d_free_external_name(&external_name(dentry)->u.head);
-
+ kfree(external_name(dentry));
kmem_cache_free(dentry_cache, dentry);
}
@@ -305,7 +291,7 @@ void release_dentry_name_snapshot(struct name_snapshot *name)
struct external_name *p;
p = container_of(name->name, struct external_name, name[0]);
if (unlikely(atomic_dec_and_test(&p->u.count)))
- call_rcu(&p->u.head, __d_free_external_name);
+ kfree_rcu(p, u.head);
}
}
EXPORT_SYMBOL(release_dentry_name_snapshot);
@@ -1608,7 +1594,6 @@ EXPORT_SYMBOL(d_invalidate);
struct dentry *__d_alloc(struct super_block *sb, const struct qstr *name)
{
- struct external_name *ext = NULL;
struct dentry *dentry;
char *dname;
int err;
@@ -1629,14 +1614,15 @@ struct dentry *__d_alloc(struct super_block *sb, const struct qstr *name)
dname = dentry->d_iname;
} else if (name->len > DNAME_INLINE_LEN-1) {
size_t size = offsetof(struct external_name, name[1]);
-
- ext = kmalloc(size + name->len, GFP_KERNEL_ACCOUNT);
- if (!ext) {
+ struct external_name *p = kmalloc(size + name->len,
+ GFP_KERNEL_ACCOUNT |
+ __GFP_RECLAIMABLE);
+ if (!p) {
kmem_cache_free(dentry_cache, dentry);
return NULL;
}
- atomic_set(&ext->u.count, 1);
- dname = ext->name;
+ atomic_set(&p->u.count, 1);
+ dname = p->name;
} else {
dname = dentry->d_iname;
}
@@ -1675,12 +1661,6 @@ struct dentry *__d_alloc(struct super_block *sb, const struct qstr *name)
}
}
- if (unlikely(ext)) {
- pg_data_t *pgdat = page_pgdat(virt_to_page(ext));
- mod_node_page_state(pgdat, NR_INDIRECTLY_RECLAIMABLE_BYTES,
- ksize(ext));
- }
-
this_cpu_inc(nr_dentry);
return dentry;
@@ -2761,7 +2741,7 @@ static void copy_name(struct dentry *dentry, struct dentry *target)
dentry->d_name.hash_len = target->d_name.hash_len;
}
if (old_name && likely(atomic_dec_and_test(&old_name->u.count)))
- call_rcu(&old_name->u.head, __d_free_external_name);
+ kfree_rcu(old_name, u.head);
}
/*
--
2.17.1
^ permalink raw reply related
* [PATCH v2 3/7] mm, slab: allocate off-slab freelists as reclaimable when appropriate
From: Vlastimil Babka @ 2018-06-18 9:18 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-kernel, Roman Gushchin, Michal Hocko,
Johannes Weiner, linux-api, Christoph Lameter, David Rientjes,
Mel Gorman, Matthew Wilcox, Vlastimil Babka
In-Reply-To: <20180618091808.4419-1-vbabka@suse.cz>
In SLAB, OFF_SLAB caches allocate management structures (currently just the
freelist) from kmalloc caches when placement in a slab page together with
objects would lead to suboptimal memory usage. For SLAB_RECLAIM_ACCOUNT caches,
we can allocate the freelists from the newly introduced reclaimable kmalloc
caches, because shrinking the OFF_SLAB cache will in general result to freeing
of the freelists as well. This should improve accounting and anti-fragmentation
a bit.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
mm/slab.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/mm/slab.c b/mm/slab.c
index 9515798f37b2..99d779ba2b92 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -2140,8 +2140,13 @@ int __kmem_cache_create(struct kmem_cache *cachep, slab_flags_t flags)
#endif
if (OFF_SLAB(cachep)) {
+ /*
+ * If this cache is reclaimable, allocate also freelists from
+ * a reclaimable kmalloc cache.
+ */
cachep->freelist_cache =
- kmalloc_slab(cachep->freelist_size, 0u);
+ kmalloc_slab(cachep->freelist_size,
+ cachep->allocflags & __GFP_RECLAIMABLE);
}
err = setup_cpu_cache(cachep, gfp);
--
2.17.1
^ permalink raw reply related
* [PATCH v2 2/7] mm, slab/slub: introduce kmalloc-reclaimable caches
From: Vlastimil Babka @ 2018-06-18 9:18 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-kernel, Roman Gushchin, Michal Hocko,
Johannes Weiner, linux-api, Christoph Lameter, David Rientjes,
Mel Gorman, Matthew Wilcox, Vlastimil Babka
In-Reply-To: <20180618091808.4419-1-vbabka@suse.cz>
Kmem caches can be created with a SLAB_RECLAIM_ACCOUNT flag, which indicates
they contain objects which can be reclaimed under memory pressure (typically
through a shrinker). This makes the slab pages accounted as NR_SLAB_RECLAIMABLE
in vmstat, which is reflected also the MemAvailable meminfo counter and in
overcommit decisions. The slab pages are also allocated with __GFP_RECLAIMABLE,
which is good for anti-fragmentation through grouping pages by mobility.
The generic kmalloc-X caches are created without this flag, but sometimes are
used also for objects that can be reclaimed, which due to varying size cannot
have a dedicated kmem cache with SLAB_RECLAIM_ACCOUNT flag. A prominent example
are dcache external names, which prompted the creation of a new, manually
managed vmstat counter NR_INDIRECTLY_RECLAIMABLE_BYTES in commit f1782c9bc547
("dcache: account external names as indirectly reclaimable memory").
To better handle this and any other similar cases, this patch introduces
SLAB_RECLAIM_ACCOUNT variants of kmalloc caches, named kmalloc-rcl-X.
They are used whenever the kmalloc() call passes __GFP_RECLAIMABLE among gfp
flags. They are added to the kmalloc_caches array as a new type. Allocations
with both __GFP_DMA and __GFP_RECLAIMABLE will use a dma type cache.
This change only applies to SLAB and SLUB, not SLOB. This is fine, since SLOB's
target are tiny system and this patch does add some overhead of kmem management
objects.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
include/linux/slab.h | 16 +++++++++++----
mm/slab_common.c | 48 ++++++++++++++++++++++++++++----------------
2 files changed, 43 insertions(+), 21 deletions(-)
diff --git a/include/linux/slab.h b/include/linux/slab.h
index 4299c59353a1..d89e934e0d8b 100644
--- a/include/linux/slab.h
+++ b/include/linux/slab.h
@@ -296,11 +296,12 @@ static inline void __check_heap_object(const void *ptr, unsigned long n,
(KMALLOC_MIN_SIZE) : 16)
#define KMALLOC_NORMAL 0
+#define KMALLOC_RECLAIM 1
#ifdef CONFIG_ZONE_DMA
-#define KMALLOC_DMA 1
-#define KMALLOC_TYPES 2
+#define KMALLOC_DMA 2
+#define KMALLOC_TYPES 3
#else
-#define KMALLOC_TYPES 1
+#define KMALLOC_TYPES 2
#endif
#ifndef CONFIG_SLOB
@@ -309,12 +310,19 @@ extern struct kmem_cache *kmalloc_caches[KMALLOC_TYPES][KMALLOC_SHIFT_HIGH + 1];
static __always_inline unsigned int kmalloc_type(gfp_t flags)
{
int is_dma = 0;
+ int is_reclaimable;
#ifdef CONFIG_ZONE_DMA
is_dma = !!(flags & __GFP_DMA);
#endif
- return is_dma;
+ is_reclaimable = !!(flags & __GFP_RECLAIMABLE);
+
+ /*
+ * If an allocation is botth __GFP_DMA and __GFP_RECLAIMABLE, return
+ * KMALLOC_DMA and effectively ignore __GFP_RECLAIMABLE
+ */
+ return (is_dma * 2) + (is_reclaimable & !is_dma);
}
/*
diff --git a/mm/slab_common.c b/mm/slab_common.c
index 635f2d8d0198..8a30d6979936 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -1103,10 +1103,21 @@ void __init setup_kmalloc_cache_index_table(void)
}
}
-static void __init new_kmalloc_cache(int idx, slab_flags_t flags)
+static void __init
+new_kmalloc_cache(int idx, int type, slab_flags_t flags)
{
- kmalloc_caches[KMALLOC_NORMAL][idx] = create_kmalloc_cache(
- kmalloc_info[idx].name,
+ const char *name;
+
+ if (type == KMALLOC_RECLAIM) {
+ flags |= SLAB_RECLAIM_ACCOUNT;
+ name = kasprintf(GFP_NOWAIT, "kmalloc-rcl-%u",
+ kmalloc_info[idx].size);
+ BUG_ON(!name);
+ } else {
+ name = kmalloc_info[idx].name;
+ }
+
+ kmalloc_caches[type][idx] = create_kmalloc_cache(name,
kmalloc_info[idx].size, flags, 0,
kmalloc_info[idx].size);
}
@@ -1118,22 +1129,25 @@ static void __init new_kmalloc_cache(int idx, slab_flags_t flags)
*/
void __init create_kmalloc_caches(slab_flags_t flags)
{
- int i;
- int type = KMALLOC_NORMAL;
+ int i, type;
- for (i = KMALLOC_SHIFT_LOW; i <= KMALLOC_SHIFT_HIGH; i++) {
- if (!kmalloc_caches[type][i])
- new_kmalloc_cache(i, flags);
+ for (type = KMALLOC_NORMAL; type <= KMALLOC_RECLAIM; type++) {
+ for (i = KMALLOC_SHIFT_LOW; i <= KMALLOC_SHIFT_HIGH; i++) {
+ if (!kmalloc_caches[type][i])
+ new_kmalloc_cache(i, type, flags);
- /*
- * Caches that are not of the two-to-the-power-of size.
- * These have to be created immediately after the
- * earlier power of two caches
- */
- if (KMALLOC_MIN_SIZE <= 32 && !kmalloc_caches[type][1] && i == 6)
- new_kmalloc_cache(1, flags);
- if (KMALLOC_MIN_SIZE <= 64 && !kmalloc_caches[type][2] && i == 7)
- new_kmalloc_cache(2, flags);
+ /*
+ * Caches that are not of the two-to-the-power-of size.
+ * These have to be created immediately after the
+ * earlier power of two caches
+ */
+ if (KMALLOC_MIN_SIZE <= 32 && i == 6 &&
+ !kmalloc_caches[type][1])
+ new_kmalloc_cache(1, type, flags);
+ if (KMALLOC_MIN_SIZE <= 64 && i == 7 &&
+ !kmalloc_caches[type][2])
+ new_kmalloc_cache(2, type, flags);
+ }
}
/* Kmalloc array is now usable */
--
2.17.1
^ permalink raw reply related
* [PATCH v2 1/7] mm, slab: combine kmalloc_caches and kmalloc_dma_caches
From: Vlastimil Babka @ 2018-06-18 9:18 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-kernel, Roman Gushchin, Michal Hocko,
Johannes Weiner, linux-api, Christoph Lameter, David Rientjes,
Mel Gorman, Matthew Wilcox, Vlastimil Babka
In-Reply-To: <20180618091808.4419-1-vbabka@suse.cz>
The kmalloc caches currently mainain separate (optional) array
kmalloc_dma_caches for __GFP_DMA allocations. There are tests for __GFP_DMA in
the allocation hotpaths. We can avoid the branches by combining kmalloc_caches
and kmalloc_dma_caches into a single two-dimensional array where the outer
dimension is cache "type". This will also allow to add kmalloc-reclaimable
caches as a third type.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
include/linux/slab.h | 41 ++++++++++++++++++++++++++++++-----------
mm/slab.c | 4 ++--
mm/slab_common.c | 30 +++++++++++-------------------
mm/slub.c | 13 +++++++------
4 files changed, 50 insertions(+), 38 deletions(-)
diff --git a/include/linux/slab.h b/include/linux/slab.h
index 14e3fe4bd6a1..4299c59353a1 100644
--- a/include/linux/slab.h
+++ b/include/linux/slab.h
@@ -295,12 +295,28 @@ static inline void __check_heap_object(const void *ptr, unsigned long n,
#define SLAB_OBJ_MIN_SIZE (KMALLOC_MIN_SIZE < 16 ? \
(KMALLOC_MIN_SIZE) : 16)
+#define KMALLOC_NORMAL 0
+#ifdef CONFIG_ZONE_DMA
+#define KMALLOC_DMA 1
+#define KMALLOC_TYPES 2
+#else
+#define KMALLOC_TYPES 1
+#endif
+
#ifndef CONFIG_SLOB
-extern struct kmem_cache *kmalloc_caches[KMALLOC_SHIFT_HIGH + 1];
+extern struct kmem_cache *kmalloc_caches[KMALLOC_TYPES][KMALLOC_SHIFT_HIGH + 1];
+
+static __always_inline unsigned int kmalloc_type(gfp_t flags)
+{
+ int is_dma = 0;
+
#ifdef CONFIG_ZONE_DMA
-extern struct kmem_cache *kmalloc_dma_caches[KMALLOC_SHIFT_HIGH + 1];
+ is_dma = !!(flags & __GFP_DMA);
#endif
+ return is_dma;
+}
+
/*
* Figure out which kmalloc slab an allocation of a certain size
* belongs to.
@@ -501,18 +517,20 @@ static __always_inline void *kmalloc_large(size_t size, gfp_t flags)
static __always_inline void *kmalloc(size_t size, gfp_t flags)
{
if (__builtin_constant_p(size)) {
+#ifndef CONFIG_SLOB
+ unsigned int index;
+#endif
if (size > KMALLOC_MAX_CACHE_SIZE)
return kmalloc_large(size, flags);
#ifndef CONFIG_SLOB
- if (!(flags & GFP_DMA)) {
- unsigned int index = kmalloc_index(size);
+ index = kmalloc_index(size);
- if (!index)
- return ZERO_SIZE_PTR;
+ if (!index)
+ return ZERO_SIZE_PTR;
- return kmem_cache_alloc_trace(kmalloc_caches[index],
- flags, size);
- }
+ return kmem_cache_alloc_trace(
+ kmalloc_caches[kmalloc_type(flags)][index],
+ flags, size);
#endif
}
return __kmalloc(size, flags);
@@ -542,13 +560,14 @@ static __always_inline void *kmalloc_node(size_t size, gfp_t flags, int node)
{
#ifndef CONFIG_SLOB
if (__builtin_constant_p(size) &&
- size <= KMALLOC_MAX_CACHE_SIZE && !(flags & GFP_DMA)) {
+ size <= KMALLOC_MAX_CACHE_SIZE) {
unsigned int i = kmalloc_index(size);
if (!i)
return ZERO_SIZE_PTR;
- return kmem_cache_alloc_node_trace(kmalloc_caches[i],
+ return kmem_cache_alloc_node_trace(
+ kmalloc_caches[kmalloc_type(flags)][i],
flags, node, size);
}
#endif
diff --git a/mm/slab.c b/mm/slab.c
index aa76a70e087e..9515798f37b2 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -1288,7 +1288,7 @@ void __init kmem_cache_init(void)
* Initialize the caches that provide memory for the kmem_cache_node
* structures first. Without this, further allocations will bug.
*/
- kmalloc_caches[INDEX_NODE] = create_kmalloc_cache(
+ kmalloc_caches[KMALLOC_NORMAL][INDEX_NODE] = create_kmalloc_cache(
kmalloc_info[INDEX_NODE].name,
kmalloc_size(INDEX_NODE), ARCH_KMALLOC_FLAGS,
0, kmalloc_size(INDEX_NODE));
@@ -1304,7 +1304,7 @@ void __init kmem_cache_init(void)
for_each_online_node(nid) {
init_list(kmem_cache, &init_kmem_cache_node[CACHE_CACHE + nid], nid);
- init_list(kmalloc_caches[INDEX_NODE],
+ init_list(kmalloc_caches[KMALLOC_NORMAL][INDEX_NODE],
&init_kmem_cache_node[SIZE_NODE + nid], nid);
}
}
diff --git a/mm/slab_common.c b/mm/slab_common.c
index 890b1f04a03a..635f2d8d0198 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -969,14 +969,9 @@ struct kmem_cache *__init create_kmalloc_cache(const char *name,
return s;
}
-struct kmem_cache *kmalloc_caches[KMALLOC_SHIFT_HIGH + 1] __ro_after_init;
+struct kmem_cache *kmalloc_caches[KMALLOC_TYPES][KMALLOC_SHIFT_HIGH + 1] __ro_after_init;
EXPORT_SYMBOL(kmalloc_caches);
-#ifdef CONFIG_ZONE_DMA
-struct kmem_cache *kmalloc_dma_caches[KMALLOC_SHIFT_HIGH + 1] __ro_after_init;
-EXPORT_SYMBOL(kmalloc_dma_caches);
-#endif
-
/*
* Conversion table for small slabs sizes / 8 to the index in the
* kmalloc array. This is necessary for slabs < 192 since we have non power
@@ -1036,12 +1031,7 @@ struct kmem_cache *kmalloc_slab(size_t size, gfp_t flags)
} else
index = fls(size - 1);
-#ifdef CONFIG_ZONE_DMA
- if (unlikely((flags & GFP_DMA)))
- return kmalloc_dma_caches[index];
-
-#endif
- return kmalloc_caches[index];
+ return kmalloc_caches[kmalloc_type(flags)][index];
}
/*
@@ -1115,7 +1105,8 @@ void __init setup_kmalloc_cache_index_table(void)
static void __init new_kmalloc_cache(int idx, slab_flags_t flags)
{
- kmalloc_caches[idx] = create_kmalloc_cache(kmalloc_info[idx].name,
+ kmalloc_caches[KMALLOC_NORMAL][idx] = create_kmalloc_cache(
+ kmalloc_info[idx].name,
kmalloc_info[idx].size, flags, 0,
kmalloc_info[idx].size);
}
@@ -1128,9 +1119,10 @@ static void __init new_kmalloc_cache(int idx, slab_flags_t flags)
void __init create_kmalloc_caches(slab_flags_t flags)
{
int i;
+ int type = KMALLOC_NORMAL;
for (i = KMALLOC_SHIFT_LOW; i <= KMALLOC_SHIFT_HIGH; i++) {
- if (!kmalloc_caches[i])
+ if (!kmalloc_caches[type][i])
new_kmalloc_cache(i, flags);
/*
@@ -1138,9 +1130,9 @@ void __init create_kmalloc_caches(slab_flags_t flags)
* These have to be created immediately after the
* earlier power of two caches
*/
- if (KMALLOC_MIN_SIZE <= 32 && !kmalloc_caches[1] && i == 6)
+ if (KMALLOC_MIN_SIZE <= 32 && !kmalloc_caches[type][1] && i == 6)
new_kmalloc_cache(1, flags);
- if (KMALLOC_MIN_SIZE <= 64 && !kmalloc_caches[2] && i == 7)
+ if (KMALLOC_MIN_SIZE <= 64 && !kmalloc_caches[type][2] && i == 7)
new_kmalloc_cache(2, flags);
}
@@ -1149,7 +1141,7 @@ void __init create_kmalloc_caches(slab_flags_t flags)
#ifdef CONFIG_ZONE_DMA
for (i = 0; i <= KMALLOC_SHIFT_HIGH; i++) {
- struct kmem_cache *s = kmalloc_caches[i];
+ struct kmem_cache *s = kmalloc_caches[KMALLOC_NORMAL][i];
if (s) {
unsigned int size = kmalloc_size(i);
@@ -1157,8 +1149,8 @@ void __init create_kmalloc_caches(slab_flags_t flags)
"dma-kmalloc-%u", size);
BUG_ON(!n);
- kmalloc_dma_caches[i] = create_kmalloc_cache(n,
- size, SLAB_CACHE_DMA | flags, 0, 0);
+ kmalloc_caches[KMALLOC_DMA][i] = create_kmalloc_cache(
+ n, size, SLAB_CACHE_DMA | flags, 0, 0);
}
}
#endif
diff --git a/mm/slub.c b/mm/slub.c
index a3b8467c14af..cdc31c1561c3 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -4659,6 +4659,7 @@ static int list_locations(struct kmem_cache *s, char *buf,
static void __init resiliency_test(void)
{
u8 *p;
+ int type = KMALLOC_NORMAL;
BUILD_BUG_ON(KMALLOC_MIN_SIZE > 16 || KMALLOC_SHIFT_HIGH < 10);
@@ -4671,7 +4672,7 @@ static void __init resiliency_test(void)
pr_err("\n1. kmalloc-16: Clobber Redzone/next pointer 0x12->0x%p\n\n",
p + 16);
- validate_slab_cache(kmalloc_caches[4]);
+ validate_slab_cache(kmalloc_caches[type][4]);
/* Hmmm... The next two are dangerous */
p = kzalloc(32, GFP_KERNEL);
@@ -4680,33 +4681,33 @@ static void __init resiliency_test(void)
p);
pr_err("If allocated object is overwritten then not detectable\n\n");
- validate_slab_cache(kmalloc_caches[5]);
+ validate_slab_cache(kmalloc_caches[type][5]);
p = kzalloc(64, GFP_KERNEL);
p += 64 + (get_cycles() & 0xff) * sizeof(void *);
*p = 0x56;
pr_err("\n3. kmalloc-64: corrupting random byte 0x56->0x%p\n",
p);
pr_err("If allocated object is overwritten then not detectable\n\n");
- validate_slab_cache(kmalloc_caches[6]);
+ validate_slab_cache(kmalloc_caches[type][6]);
pr_err("\nB. Corruption after free\n");
p = kzalloc(128, GFP_KERNEL);
kfree(p);
*p = 0x78;
pr_err("1. kmalloc-128: Clobber first word 0x78->0x%p\n\n", p);
- validate_slab_cache(kmalloc_caches[7]);
+ validate_slab_cache(kmalloc_caches[type][7]);
p = kzalloc(256, GFP_KERNEL);
kfree(p);
p[50] = 0x9a;
pr_err("\n2. kmalloc-256: Clobber 50th byte 0x9a->0x%p\n\n", p);
- validate_slab_cache(kmalloc_caches[8]);
+ validate_slab_cache(kmalloc_caches[type][8]);
p = kzalloc(512, GFP_KERNEL);
kfree(p);
p[512] = 0xab;
pr_err("\n3. kmalloc-512: Clobber redzone 0xab->0x%p\n\n", p);
- validate_slab_cache(kmalloc_caches[9]);
+ validate_slab_cache(kmalloc_caches[type][9]);
}
#else
#ifdef CONFIG_SYSFS
--
2.17.1
^ permalink raw reply related
* [PATCH v2 0/7] kmalloc-reclaimable caches
From: Vlastimil Babka @ 2018-06-18 9:18 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-kernel, Roman Gushchin, Michal Hocko,
Johannes Weiner, linux-api, Christoph Lameter, David Rientjes,
Mel Gorman, Matthew Wilcox, Vlastimil Babka, Laura Abbott,
Sumit Semwal, Vijayanand Jitta
v2 changes:
- shorten cache names to kmalloc-rcl-<SIZE>
- last patch shortens <SIZE> for all kmalloc caches to e.g. "1k", "4M"
- include dma caches to the 2D kmalloc_caches[] array to avoid a branch
- vmstat counter nr_indirectly_reclaimable_bytes renamed to
nr_kernel_misc_reclaimable, doesn't include kmalloc-rcl-*
- /proc/meminfo counter renamed to KReclaimable, includes kmalloc-rcl*
and nr_kernel_misc_reclaimable
Hi,
as discussed at LSF/MM [1] here's a patchset that introduces
kmalloc-reclaimable caches (more details in the second patch) and uses them for
SLAB freelists and dcache external names. The latter allows us to repurpose the
NR_INDIRECTLY_RECLAIMABLE_BYTES counter later in the series.
This is how /proc/slabinfo looks like after booting in virtme:
...
kmalloc-rcl-4M 0 0 4194304 1 1024 : tunables 1 1 0 : slabdata 0 0 0
...
kmalloc-rcl-96 7 32 128 32 1 : tunables 120 60 8 : slabdata 1 1 0
kmalloc-rcl-64 25 128 64 64 1 : tunables 120 60 8 : slabdata 2 2 0
kmalloc-rcl-32 0 0 32 124 1 : tunables 120 60 8 : slabdata 0 0 0
kmalloc-4M 0 0 4194304 1 1024 : tunables 1 1 0 : slabdata 0 0 0
kmalloc-2M 0 0 2097152 1 512 : tunables 1 1 0 : slabdata 0 0 0
kmalloc-1M 0 0 1048576 1 256 : tunables 1 1 0 : slabdata 0 0 0
...
/proc/vmstat with renamed nr_indirectly_reclaimable_bytes counter:
...
nr_slab_reclaimable 2817
nr_slab_unreclaimable 1781
...
nr_kernel_misc_reclaimable 0
...
/proc/meminfo with new KReclaimable counter:
...
Shmem: 564 kB
KReclaimable: 11260 kB
Slab: 18368 kB
SReclaimable: 11260 kB
SUnreclaim: 7108 kB
KernelStack: 1248 kB
...
Thanks,
Vlastimil
Vlastimil Babka (7):
mm, slab: combine kmalloc_caches and kmalloc_dma_caches
mm, slab/slub: introduce kmalloc-reclaimable caches
mm, slab: allocate off-slab freelists as reclaimable when appropriate
dcache: allocate external names from reclaimable kmalloc caches
mm: rename and change semantics of nr_indirectly_reclaimable_bytes
mm, proc: add KReclaimable to /proc/meminfo
mm, slab: shorten kmalloc cache names for large sizes
Documentation/filesystems/proc.txt | 4 +
drivers/base/node.c | 19 ++--
drivers/staging/android/ion/ion_page_pool.c | 4 +-
fs/dcache.c | 38 ++------
fs/proc/meminfo.c | 16 +--
include/linux/mmzone.h | 2 +-
include/linux/slab.h | 49 +++++++---
mm/page_alloc.c | 19 ++--
mm/slab.c | 11 ++-
mm/slab_common.c | 102 ++++++++++++--------
mm/slub.c | 13 +--
mm/util.c | 3 +-
mm/vmstat.c | 6 +-
13 files changed, 159 insertions(+), 127 deletions(-)
--
2.17.1
^ permalink raw reply
* [PATCH 3/3] time: Change types to new y2038 safe __kernel_itimerspec
From: Deepa Dinamani @ 2018-06-17 5:11 UTC (permalink / raw)
To: arnd, tglx, linux-kernel; +Cc: viro, linux-fsdevel, linux-api, y2038
In-Reply-To: <20180617051144.29756-1-deepa.kernel@gmail.com>
timer_set/gettime and timerfd_set/get apis use struct itimerspec at the
user interface layer. struct itimerspec is not y2038-safe.
Change these interfaces to use y2038-safe struct __kernel_itimerspec
instead. This will help define new syscalls when architectures select
CONFIG_64BIT_TIME.
Signed-off-by: Deepa Dinamani <deepa.kernel@gmail.com>
---
fs/timerfd.c | 8 ++++----
include/linux/syscalls.h | 10 +++++-----
kernel/time/posix-timers.c | 12 +++++++-----
3 files changed, 16 insertions(+), 14 deletions(-)
diff --git a/fs/timerfd.c b/fs/timerfd.c
index d84a2bee4f82..8bb926253f88 100644
--- a/fs/timerfd.c
+++ b/fs/timerfd.c
@@ -533,8 +533,8 @@ static int do_timerfd_gettime(int ufd, struct itimerspec64 *t)
}
SYSCALL_DEFINE4(timerfd_settime, int, ufd, int, flags,
- const struct itimerspec __user *, utmr,
- struct itimerspec __user *, otmr)
+ const struct __kernel_itimerspec __user *, utmr,
+ struct __kernel_itimerspec __user *, otmr)
{
struct itimerspec64 new, old;
int ret;
@@ -550,7 +550,7 @@ SYSCALL_DEFINE4(timerfd_settime, int, ufd, int, flags,
return ret;
}
-SYSCALL_DEFINE2(timerfd_gettime, int, ufd, struct itimerspec __user *, otmr)
+SYSCALL_DEFINE2(timerfd_gettime, int, ufd, struct __kernel_itimerspec __user *, otmr)
{
struct itimerspec64 kotmr;
int ret = do_timerfd_gettime(ufd, &kotmr);
@@ -559,7 +559,7 @@ SYSCALL_DEFINE2(timerfd_gettime, int, ufd, struct itimerspec __user *, otmr)
return put_itimerspec64(&kotmr, otmr) ? -EFAULT : 0;
}
-#ifdef CONFIG_COMPAT
+#ifdef CONFIG_COMPAT_32BIT_TIME
COMPAT_SYSCALL_DEFINE4(timerfd_settime, int, ufd, int, flags,
const struct compat_itimerspec __user *, utmr,
struct compat_itimerspec __user *, otmr)
diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index 73810808cdf2..38b9ec152024 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -501,9 +501,9 @@ asmlinkage long sys_sync_file_range(int fd, loff_t offset, loff_t nbytes,
/* fs/timerfd.c */
asmlinkage long sys_timerfd_create(int clockid, int flags);
asmlinkage long sys_timerfd_settime(int ufd, int flags,
- const struct itimerspec __user *utmr,
- struct itimerspec __user *otmr);
-asmlinkage long sys_timerfd_gettime(int ufd, struct itimerspec __user *otmr);
+ const struct __kernel_itimerspec __user *utmr,
+ struct __kernel_itimerspec __user *otmr);
+asmlinkage long sys_timerfd_gettime(int ufd, struct __kernel_itimerspec __user *otmr);
/* fs/utimes.c */
asmlinkage long sys_utimensat(int dfd, const char __user *filename,
@@ -568,10 +568,10 @@ asmlinkage long sys_timer_create(clockid_t which_clock,
struct sigevent __user *timer_event_spec,
timer_t __user * created_timer_id);
asmlinkage long sys_timer_gettime(timer_t timer_id,
- struct itimerspec __user *setting);
+ struct __kernel_itimerspec __user *setting);
asmlinkage long sys_timer_getoverrun(timer_t timer_id);
asmlinkage long sys_timer_settime(timer_t timer_id, int flags,
- const struct itimerspec __user *new_setting,
+ const struct __kernel_itimerspec __user *new_setting,
struct itimerspec __user *old_setting);
asmlinkage long sys_timer_delete(timer_t timer_id);
asmlinkage long sys_clock_settime(clockid_t which_clock,
diff --git a/kernel/time/posix-timers.c b/kernel/time/posix-timers.c
index e08ce3f27447..bc35603cf2a9 100644
--- a/kernel/time/posix-timers.c
+++ b/kernel/time/posix-timers.c
@@ -743,7 +743,7 @@ static int do_timer_gettime(timer_t timer_id, struct itimerspec64 *setting)
/* Get the time remaining on a POSIX.1b interval timer. */
SYSCALL_DEFINE2(timer_gettime, timer_t, timer_id,
- struct itimerspec __user *, setting)
+ struct __kernel_itimerspec __user *, setting)
{
struct itimerspec64 cur_setting;
@@ -755,7 +755,8 @@ SYSCALL_DEFINE2(timer_gettime, timer_t, timer_id,
return ret;
}
-#ifdef CONFIG_COMPAT
+#ifdef CONFIG_COMPAT_32BIT_TIME
+
COMPAT_SYSCALL_DEFINE2(timer_gettime, timer_t, timer_id,
struct compat_itimerspec __user *, setting)
{
@@ -768,6 +769,7 @@ COMPAT_SYSCALL_DEFINE2(timer_gettime, timer_t, timer_id,
}
return ret;
}
+
#endif
/*
@@ -906,8 +908,8 @@ static int do_timer_settime(timer_t timer_id, int flags,
/* Set a POSIX.1b interval timer */
SYSCALL_DEFINE4(timer_settime, timer_t, timer_id, int, flags,
- const struct itimerspec __user *, new_setting,
- struct itimerspec __user *, old_setting)
+ const struct __kernel_itimerspec __user *, new_setting,
+ struct __kernel_itimerspec __user *, old_setting)
{
struct itimerspec64 new_spec, old_spec;
struct itimerspec64 *rtn = old_setting ? &old_spec : NULL;
@@ -927,7 +929,7 @@ SYSCALL_DEFINE4(timer_settime, timer_t, timer_id, int, flags,
return error;
}
-#ifdef CONFIG_COMPAT
+#ifdef CONFIG_COMPAT_32BIT_TIME
COMPAT_SYSCALL_DEFINE4(timer_settime, timer_t, timer_id, int, flags,
struct compat_itimerspec __user *, new,
struct compat_itimerspec __user *, old)
--
2.17.1
^ permalink raw reply related
* [PATCH 2/3] time: Enable get/put_compat_itimerspec64 always
From: Deepa Dinamani @ 2018-06-17 5:11 UTC (permalink / raw)
To: arnd, tglx, linux-kernel; +Cc: linux-fsdevel, linux-api, viro, y2038
In-Reply-To: <20180617051144.29756-1-deepa.kernel@gmail.com>
This will aid in enabling the compat syscalls on 32-bit
architectures later on.
Also move compat_itimerspec and related defines to compat_time.h.
The compat_time.h file will eventually be deleted.
Signed-off-by: Deepa Dinamani <deepa.kernel@gmail.com>
---
include/linux/compat.h | 9 ---------
include/linux/compat_time.h | 9 +++++++++
kernel/compat.c | 29 -----------------------------
kernel/time/time.c | 21 +++++++++++++++++++++
4 files changed, 30 insertions(+), 38 deletions(-)
diff --git a/include/linux/compat.h b/include/linux/compat.h
index b1a5562b3215..18a13f2df14b 100644
--- a/include/linux/compat.h
+++ b/include/linux/compat.h
@@ -109,11 +109,6 @@ typedef compat_ulong_t compat_aio_context_t;
struct compat_sel_arg_struct;
struct rusage;
-struct compat_itimerspec {
- struct compat_timespec it_interval;
- struct compat_timespec it_value;
-};
-
struct compat_utimbuf {
compat_time_t actime;
compat_time_t modtime;
@@ -294,10 +289,6 @@ extern int compat_get_timespec(struct timespec *, const void __user *);
extern int compat_put_timespec(const struct timespec *, void __user *);
extern int compat_get_timeval(struct timeval *, const void __user *);
extern int compat_put_timeval(const struct timeval *, void __user *);
-extern int get_compat_itimerspec64(struct itimerspec64 *its,
- const struct compat_itimerspec __user *uits);
-extern int put_compat_itimerspec64(const struct itimerspec64 *its,
- struct compat_itimerspec __user *uits);
struct compat_iovec {
compat_uptr_t iov_base;
diff --git a/include/linux/compat_time.h b/include/linux/compat_time.h
index 31f2774f1994..e70bfd1d2c3f 100644
--- a/include/linux/compat_time.h
+++ b/include/linux/compat_time.h
@@ -17,7 +17,16 @@ struct compat_timeval {
s32 tv_usec;
};
+struct compat_itimerspec {
+ struct compat_timespec it_interval;
+ struct compat_timespec it_value;
+};
+
extern int compat_get_timespec64(struct timespec64 *, const void __user *);
extern int compat_put_timespec64(const struct timespec64 *, void __user *);
+extern int get_compat_itimerspec64(struct itimerspec64 *its,
+ const struct compat_itimerspec __user *uits);
+extern int put_compat_itimerspec64(const struct itimerspec64 *its,
+ struct compat_itimerspec __user *uits);
#endif /* _LINUX_COMPAT_TIME_H */
diff --git a/kernel/compat.c b/kernel/compat.c
index 702aa846ddac..8e40efc2928a 100644
--- a/kernel/compat.c
+++ b/kernel/compat.c
@@ -324,35 +324,6 @@ COMPAT_SYSCALL_DEFINE3(sched_getaffinity, compat_pid_t, pid, unsigned int, len,
return ret;
}
-/* Todo: Delete these extern declarations when get/put_compat_itimerspec64()
- * are moved to kernel/time/time.c .
- */
-extern int __compat_get_timespec64(struct timespec64 *ts64,
- const struct compat_timespec __user *cts);
-extern int __compat_put_timespec64(const struct timespec64 *ts64,
- struct compat_timespec __user *cts);
-
-int get_compat_itimerspec64(struct itimerspec64 *its,
- const struct compat_itimerspec __user *uits)
-{
-
- if (__compat_get_timespec64(&its->it_interval, &uits->it_interval) ||
- __compat_get_timespec64(&its->it_value, &uits->it_value))
- return -EFAULT;
- return 0;
-}
-EXPORT_SYMBOL_GPL(get_compat_itimerspec64);
-
-int put_compat_itimerspec64(const struct itimerspec64 *its,
- struct compat_itimerspec __user *uits)
-{
- if (__compat_put_timespec64(&its->it_interval, &uits->it_interval) ||
- __compat_put_timespec64(&its->it_value, &uits->it_value))
- return -EFAULT;
- return 0;
-}
-EXPORT_SYMBOL_GPL(put_compat_itimerspec64);
-
/*
* We currently only need the following fields from the sigevent
* structure: sigev_value, sigev_signo, sig_notify and (sometimes
diff --git a/kernel/time/time.c b/kernel/time/time.c
index 4e65916e3d13..c7342e313886 100644
--- a/kernel/time/time.c
+++ b/kernel/time/time.c
@@ -957,3 +957,24 @@ int put_itimerspec64(const struct itimerspec64 *it,
return ret;
}
EXPORT_SYMBOL_GPL(put_itimerspec64);
+
+int get_compat_itimerspec64(struct itimerspec64 *its,
+ const struct compat_itimerspec __user *uits)
+{
+
+ if (__compat_get_timespec64(&its->it_interval, &uits->it_interval) ||
+ __compat_get_timespec64(&its->it_value, &uits->it_value))
+ return -EFAULT;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(get_compat_itimerspec64);
+
+int put_compat_itimerspec64(const struct itimerspec64 *its,
+ struct compat_itimerspec __user *uits)
+{
+ if (__compat_put_timespec64(&its->it_interval, &uits->it_interval) ||
+ __compat_put_timespec64(&its->it_value, &uits->it_value))
+ return -EFAULT;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(put_compat_itimerspec64);
--
2.17.1
_______________________________________________
Y2038 mailing list
Y2038@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/y2038
^ permalink raw reply related
* [PATCH 1/3] time: Introduce struct __kernel_itimerspec
From: Deepa Dinamani @ 2018-06-17 5:11 UTC (permalink / raw)
To: arnd, tglx, linux-kernel; +Cc: viro, linux-fsdevel, linux-api, y2038
In-Reply-To: <20180617051144.29756-1-deepa.kernel@gmail.com>
struct itimerspec is not y2038-safe.
Introduce a new struct __kernel_itimerspec based on the
kernel internal y2038-safe struct itimerspec64.
The definition of struct __kernel_itimerspec includes two
struct __kernel_timespec.
And, since struct __kernel_timespec has the same representation in native
and compat modes, so does struct __kernel_itimerspec. This helps have a
common entry point for syscalls using struct __kernel_itimerspec.
New y2038-safe syscalls will use this new type. Since most of the new
syscalls are just an update to the native syscalls with the type update,
place the new definition under CONFIG_64BIT_TIME. This helps
architectures that do not support the above config to keep using the
old definition of struct itimerspec.
Also change the get/put_itimerspec64 to use struct__kernel_itimerspec.
This will help 32 bit architectures to use the new syscalls when
architectures select CONFIG_64BIT_TIME.
Signed-off-by: Deepa Dinamani <deepa.kernel@gmail.com>
---
include/linux/time.h | 4 ++--
include/linux/time64.h | 1 +
include/uapi/linux/time.h | 7 +++++++
kernel/time/time.c | 4 ++--
4 files changed, 12 insertions(+), 4 deletions(-)
diff --git a/include/linux/time.h b/include/linux/time.h
index aed74463592d..27d83fd2ae61 100644
--- a/include/linux/time.h
+++ b/include/linux/time.h
@@ -14,9 +14,9 @@ int get_timespec64(struct timespec64 *ts,
int put_timespec64(const struct timespec64 *ts,
struct __kernel_timespec __user *uts);
int get_itimerspec64(struct itimerspec64 *it,
- const struct itimerspec __user *uit);
+ const struct __kernel_itimerspec __user *uit);
int put_itimerspec64(const struct itimerspec64 *it,
- struct itimerspec __user *uit);
+ struct __kernel_itimerspec __user *uit);
extern time64_t mktime64(const unsigned int year, const unsigned int mon,
const unsigned int day, const unsigned int hour,
diff --git a/include/linux/time64.h b/include/linux/time64.h
index 0a7b2f79cec7..05634afba0db 100644
--- a/include/linux/time64.h
+++ b/include/linux/time64.h
@@ -12,6 +12,7 @@ typedef __u64 timeu64_t;
*/
#ifndef CONFIG_64BIT_TIME
#define __kernel_timespec timespec
+#define __kernel_itimerspec itimerspec
#endif
#include <uapi/linux/time.h>
diff --git a/include/uapi/linux/time.h b/include/uapi/linux/time.h
index fcf936656493..6b56a2208be7 100644
--- a/include/uapi/linux/time.h
+++ b/include/uapi/linux/time.h
@@ -49,6 +49,13 @@ struct __kernel_timespec {
};
#endif
+#ifndef __kernel_itimerspec
+struct __kernel_itimerspec {
+ struct __kernel_timespec it_interval; /* timer period */
+ struct __kernel_timespec it_value; /* timer expiration */
+};
+#endif
+
/*
* legacy timeval structure, only embedded in structures that
* traditionally used 'timeval' to pass time intervals (not absolute
diff --git a/kernel/time/time.c b/kernel/time/time.c
index 6fa99213fc72..4e65916e3d13 100644
--- a/kernel/time/time.c
+++ b/kernel/time/time.c
@@ -929,7 +929,7 @@ int compat_put_timespec64(const struct timespec64 *ts, void __user *uts)
EXPORT_SYMBOL_GPL(compat_put_timespec64);
int get_itimerspec64(struct itimerspec64 *it,
- const struct itimerspec __user *uit)
+ const struct __kernel_itimerspec __user *uit)
{
int ret;
@@ -944,7 +944,7 @@ int get_itimerspec64(struct itimerspec64 *it,
EXPORT_SYMBOL_GPL(get_itimerspec64);
int put_itimerspec64(const struct itimerspec64 *it,
- struct itimerspec __user *uit)
+ struct __kernel_itimerspec __user *uit)
{
int ret;
--
2.17.1
^ permalink raw reply related
* [PATCH 0/3] time: Add y2038 safe struct __kernel_itimerspec
From: Deepa Dinamani @ 2018-06-17 5:11 UTC (permalink / raw)
To: arnd, tglx, linux-kernel; +Cc: linux-fsdevel, linux-api, viro, y2038
The series aims at adding a new y2038-safe struct __kernel_itimerspec.
This is intended to replace the struct itimerspec at ABI level.
The series is a continuation of efforts to convert all syscalls with
time_t or time_t-derived structures to be y2038-safe.
Arnd, maybe this series can go along with the rest of syscalls that you
have in your y2038 tree?
Deepa Dinamani (3):
time: Introduce struct __kernel_itimerspec
time: Enable get/put_compat_itimerspec64 always
time: Change types to new y2038 safe __kernel_itimerspec
fs/timerfd.c | 8 ++++----
include/linux/compat.h | 9 ---------
include/linux/compat_time.h | 9 +++++++++
include/linux/syscalls.h | 10 +++++-----
include/linux/time.h | 4 ++--
include/linux/time64.h | 1 +
include/uapi/linux/time.h | 7 +++++++
kernel/compat.c | 29 -----------------------------
kernel/time/posix-timers.c | 12 +++++++-----
kernel/time/time.c | 25 +++++++++++++++++++++++--
10 files changed, 58 insertions(+), 56 deletions(-)
base-commit: 4b373f94fee5acf2ff4c1efbb3f702060379df1f
--
2.17.1
_______________________________________________
Y2038 mailing list
Y2038@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/y2038
^ permalink raw reply
* [RFC PATCH for 4.18] rseq: cleanup: no need to abort rseq c.s. in child on fork
From: Mathieu Desnoyers @ 2018-06-16 20:20 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, linux-api, Peter Zijlstra, Paul E . McKenney,
Boqun Feng, Andy Lutomirski, Dave Watson, Paul Turner,
Andrew Morton, Russell King, Ingo Molnar, H . Peter Anvin,
Andi Kleen, Chris Lameter, Ben Maurer, Steven Rostedt,
Josh Triplett, Linus Torvalds, Catalin Marinas, Will Deacon,
Michael Kerrisk
Considering that we explicitly forbid system calls in rseq critical
sections, it is not valid to issue a fork or clone system call within a
rseq critical section, so rseq_fork() is not required to restart an
active rseq c.s. in the child process.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Joel Fernandes <joelaf@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dave Watson <davejwatson@fb.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Shuah Khan <shuahkh@osg.samsung.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: linux-kselftest@vger.kernel.org
Cc: "H . Peter Anvin" <hpa@zytor.com>
Cc: Chris Lameter <cl@linux.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Michael Kerrisk <mtk.manpages@gmail.com>
Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Paul Turner <pjt@google.com>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ben Maurer <bmaurer@fb.com>
Cc: linux-api@vger.kernel.org
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
---
include/linux/sched.h | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 87bf02d93a27..c1882643d455 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1831,9 +1831,7 @@ static inline void rseq_migrate(struct task_struct *t)
/*
* If parent process has a registered restartable sequences area, the
- * child inherits. Only applies when forking a process, not a thread. In
- * case a parent fork() in the middle of a restartable sequence, set the
- * resume notifier to force the child to retry.
+ * child inherits. Only applies when forking a process, not a thread.
*/
static inline void rseq_fork(struct task_struct *t, unsigned long clone_flags)
{
@@ -1847,7 +1845,6 @@ static inline void rseq_fork(struct task_struct *t, unsigned long clone_flags)
t->rseq_len = current->rseq_len;
t->rseq_sig = current->rseq_sig;
t->rseq_event_mask = current->rseq_event_mask;
- rseq_preempt(t);
}
}
--
2.11.0
^ permalink raw reply related
* [RFC PATCH man-pages] Add rseq manpage
From: Mathieu Desnoyers @ 2018-06-16 19:58 UTC (permalink / raw)
To: Michael Kerrisk
Cc: linux-kernel, linux-api, Peter Zijlstra, Paul E . McKenney,
Boqun Feng, Andy Lutomirski, Dave Watson, Paul Turner,
Andrew Morton, Russell King, Thomas Gleixner, Ingo Molnar,
H . Peter Anvin, Andrew Hunter, Andi Kleen, Chris Lameter,
Ben Maurer, Steven Rostedt, Josh Triplett, Linus Torvalds,
Catalin Marinas
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Paul Turner <pjt@google.com>
CC: Thomas Gleixner <tglx@linutronix.de>
CC: Andrew Hunter <ahh@google.com>
CC: Andy Lutomirski <luto@amacapital.net>
CC: Andi Kleen <andi@firstfloor.org>
CC: Dave Watson <davejwatson@fb.com>
CC: Chris Lameter <cl@linux.com>
CC: Ingo Molnar <mingo@redhat.com>
CC: "H. Peter Anvin" <hpa@zytor.com>
CC: Ben Maurer <bmaurer@fb.com>
CC: Steven Rostedt <rostedt@goodmis.org>
CC: Josh Triplett <josh@joshtriplett.org>
CC: Linus Torvalds <torvalds@linux-foundation.org>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: Russell King <linux@arm.linux.org.uk>
CC: Catalin Marinas <catalin.marinas@arm.com>
CC: Will Deacon <will.deacon@arm.com>
CC: Michael Kerrisk <mtk.manpages@gmail.com>
CC: Boqun Feng <boqun.feng@gmail.com>
CC: linux-api@vger.kernel.org
---
man2/rseq.2 | 271 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 271 insertions(+)
create mode 100644 man2/rseq.2
diff --git a/man2/rseq.2 b/man2/rseq.2
new file mode 100644
index 000000000..ab2519f82
--- /dev/null
+++ b/man2/rseq.2
@@ -0,0 +1,271 @@
+.\" Copyright 2015-2017 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
+.\"
+.\" %%%LICENSE_START(VERBATIM)
+.\" Permission is granted to make and distribute verbatim copies of this
+.\" manual provided the copyright notice and this permission notice are
+.\" preserved on all copies.
+.\"
+.\" Permission is granted to copy and distribute modified versions of this
+.\" manual under the conditions for verbatim copying, provided that the
+.\" entire resulting derived work is distributed under the terms of a
+.\" permission notice identical to this one.
+.\"
+.\" Since the Linux kernel and libraries are constantly changing, this
+.\" manual page may be incorrect or out-of-date. The author(s) assume no
+.\" responsibility for errors or omissions, or for damages resulting from
+.\" the use of the information contained herein. The author(s) may not
+.\" have taken the same level of care in the production of this manual,
+.\" which is licensed free of charge, as they might when working
+.\" professionally.
+.\"
+.\" Formatted or processed versions of this manual, if unaccompanied by
+.\" the source, must acknowledge the copyright and authors of this work.
+.\" %%%LICENSE_END
+.\"
+.TH RSEQ 2 2018-06-16 "Linux" "Linux Programmer's Manual"
+.SH NAME
+rseq \- Restartable sequences and cpu number cache
+.SH SYNOPSIS
+.nf
+.B #include <linux/rseq.h>
+.sp
+.BI "int rseq(struct rseq * " rseq ", uint32_t " rseq_len ", int " flags ", uint32_t " sig ");
+.sp
+.SH DESCRIPTION
+The
+.BR rseq ()
+ABI accelerates user-space operations on per-cpu data by defining a
+shared data structure ABI between each user-space thread and the kernel.
+
+It allows user-space to perform update operations on per-cpu data
+without requiring heavy-weight atomic operations.
+
+The term CPU used in this documentation refers to a hardware execution
+context.
+
+Restartable sequences are atomic with respect to preemption (making it
+atomic with respect to other threads running on the same CPU), as well
+as signal delivery (user-space execution contexts nested over the same
+thread).
+
+It is suited for update operations on per-cpu data.
+
+It can be used on data structures shared between threads within a
+process, and on data structures shared between threads across different
+processes.
+
+.PP
+Some examples of operations that can be accelerated or improved
+by this ABI:
+.IP \[bu] 2
+Memory allocator per-cpu free-lists,
+.IP \[bu] 2
+Querying the current CPU number,
+.IP \[bu] 2
+Incrementing per-CPU counters,
+.IP \[bu] 2
+Modifying data protected by per-CPU spinlocks,
+.IP \[bu] 2
+Inserting/removing elements in per-CPU linked-lists,
+.IP \[bu] 2
+Writing/reading per-CPU ring buffers content.
+.IP \[bu] 2
+Accurately reading performance monitoring unit counters
+with respect to thread migration.
+
+.PP
+Restartable sequences must not perform system calls. Doing so may result
+in termination of the process by a segmentation fault.
+
+.PP
+The
+.I rseq
+argument is a pointer to the thread-local rseq structure to be shared
+between kernel and user-space.
+
+.PP
+The layout of
+.B struct rseq
+is as follows:
+.TP
+.B Structure alignment
+This structure is aligned on multiples of 32 bytes.
+.TP
+.B Structure size
+This structure is extensible. Its size is passed as parameter to the
+rseq system call.
+.TP
+.B Fields
+
+.TP
+.in +4n
+.I cpu_id_start
+Optimistic cache of the CPU number on which the current thread is
+running. Its value is guaranteed to always be a possible CPU number,
+even when rseq is not initialized. The value it contains should always
+be confirmed by reading the cpu_id field.
+.in
+.TP
+.in +4n
+.I cpu_id
+Cache of the CPU number on which the current thread is running.
+-1 if uninitialized.
+.in
+.TP
+.in +4n
+.I rseq_cs
+The rseq_cs field is a pointer to a struct rseq_cs. Is is NULL when no
+rseq assembly block critical section is active for the current thread.
+Setting it to point to a critical section descriptor (struct rseq_cs)
+marks the beginning of the critical section.
+.in
+.TP
+.in +4n
+.I flags
+Flags indicating the restart behavior for the current thread. This is
+mainly used for debugging purposes. Can be either:
+.IP \[bu]
+RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT
+.IP \[bu]
+RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL
+.IP \[bu]
+RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE
+.in
+
+.PP
+The layout of
+.B struct rseq_cs
+version 0 is as follows:
+.TP
+.B Structure alignment
+This structure is aligned on multiples of 32 bytes.
+.TP
+.B Structure size
+This structure has a fixed size of 32 bytes.
+.TP
+.B Fields
+
+.TP
+.in +4n
+.I version
+Version of this structure.
+.in
+.TP
+.in +4n
+.I flags
+Flags indicating the restart behavior of this structure. Can be
+either:
+.IP \[bu]
+RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT
+.IP \[bu]
+RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL
+.IP \[bu]
+RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE
+.TP
+.in +4n
+.I start_ip
+Instruction pointer address of the first instruction of the sequence of
+consecutive assembly instructions.
+.in
+.TP
+.in +4n
+.I post_commit_offset
+Offset (from start_ip address) of the address after the last instruction
+of the sequence of consecutive assembly instructions.
+.in
+.TP
+.in +4n
+.I abort_ip
+Instruction pointer address where to move the execution flow in case of
+abort of the sequence of consecutive assembly instructions.
+.in
+
+.PP
+The
+.I rseq_len
+argument is the size of the
+.I struct rseq
+to register.
+
+.PP
+The
+.I flags
+argument is 0 for registration, and
+.IR RSEQ_FLAG_UNREGISTER
+for unregistration.
+
+.PP
+The
+.I sig
+argument is the 32-bit signature to be expected before the abort
+handler code.
+
+.PP
+A single library per process should keep the rseq structure in a
+thread-local storage variable.
+The
+.I cpu_id
+field should be initialized to -1, and the
+.I cpu_id_start
+field should be initialized to a possible CPU value (typically 0).
+
+.PP
+Each thread is responsible for registering and unregistering its rseq
+structure. No more than one rseq structure address can be registered
+per thread at a given time.
+
+.PP
+In a typical usage scenario, the thread registering the rseq
+structure will be performing loads and stores from/to that structure. It
+is however also allowed to read that structure from other threads.
+The rseq field updates performed by the kernel provide relaxed atomicity
+semantics, which guarantee that other threads performing relaxed atomic
+reads of the cpu number cache will always observe a consistent value.
+
+.SH RETURN VALUE
+A return value of 0 indicates success. On error, \-1 is returned, and
+.I errno
+is set appropriately.
+
+.SH ERRORS
+.TP
+.B EINVAL
+Either
+.I flags
+contains an invalid value, or
+.I rseq
+contains an address which is not appropriately aligned, or
+.I rseq_len
+contains a size that does not match the size received on registration.
+.TP
+.B ENOSYS
+The
+.BR rseq ()
+system call is not implemented by this kernel.
+.TP
+.B EFAULT
+.I rseq
+is an invalid address.
+.TP
+.B EBUSY
+Restartable sequence is already registered for this thread.
+.TP
+.B EPERM
+The
+.I sig
+argument on unregistration does not match the signature received
+on registration.
+
+.SH VERSIONS
+The
+.BR rseq ()
+system call was added in Linux 4.18.
+
+.SH CONFORMING TO
+.BR rseq ()
+is Linux-specific.
+
+.in
+.SH SEE ALSO
+.BR sched_getcpu (3) ,
+.BR membarrier (2)
--
2.11.0
^ permalink raw reply related
* [RFC PATCH for 4.18 2/2] rseq/selftests: arm: align struct rseq_cs on 32 bytes
From: Mathieu Desnoyers @ 2018-06-16 19:44 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, Mathieu Desnoyers, Joel Fernandes, Peter Zijlstra,
Catalin Marinas, Dave Watson, Will Deacon, Shuah Khan, Andi Kleen,
linux-kselftest, H . Peter Anvin, Chris Lameter, Russell King,
Andrew Hunter, Michael Kerrisk, Paul E . McKenney, Paul Turner,
Boqun Feng, Josh Triplett, Steven Rostedt
In-Reply-To: <20180616194456.26159-1-mathieu.desnoyers@efficios.com>
uapi/linux/rseq.h aligns struct rseq_cs on 32 bytes. Satisfy this
alignment requirement in its definition within the rseq-arm.h
inline assembly.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Joel Fernandes <joelaf@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dave Watson <davejwatson@fb.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Shuah Khan <shuahkh@osg.samsung.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: linux-kselftest@vger.kernel.org
Cc: "H . Peter Anvin" <hpa@zytor.com>
Cc: Chris Lameter <cl@linux.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Andrew Hunter <ahh@google.com>
Cc: Michael Kerrisk <mtk.manpages@gmail.com>
Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Paul Turner <pjt@google.com>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ben Maurer <bmaurer@fb.com>
Cc: linux-api@vger.kernel.org
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
---
tools/testing/selftests/rseq/rseq-arm.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/testing/selftests/rseq/rseq-arm.h b/tools/testing/selftests/rseq/rseq-arm.h
index 3b055f9aeaab..3cea19877227 100644
--- a/tools/testing/selftests/rseq/rseq-arm.h
+++ b/tools/testing/selftests/rseq/rseq-arm.h
@@ -57,6 +57,7 @@ do { \
#define __RSEQ_ASM_DEFINE_ABORT(table_label, label, teardown, \
abort_label, version, flags, \
start_ip, post_commit_offset, abort_ip) \
+ ".balign 32\n\t" \
__rseq_str(table_label) ":\n\t" \
".word " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \
".word " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) ", 0x0\n\t" \
--
2.11.0
^ permalink raw reply related
* [RFC PATCH for 4.18 1/2] rseq/selftests: run_param_test.sh should be executable
From: Mathieu Desnoyers @ 2018-06-16 19:44 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, Mathieu Desnoyers, Joel Fernandes, Peter Zijlstra,
Catalin Marinas, Dave Watson, Will Deacon, Shuah Khan, Andi Kleen,
linux-kselftest, H . Peter Anvin, Chris Lameter, Russell King,
Andrew Hunter, Michael Kerrisk, Paul E . McKenney, Paul Turner,
Boqun Feng, Josh Triplett, Steven Rostedt
In-Reply-To: <20180616194456.26159-1-mathieu.desnoyers@efficios.com>
The executable bit of the run_param_test.sh script got lost in
the merge.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Joel Fernandes <joelaf@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dave Watson <davejwatson@fb.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Shuah Khan <shuahkh@osg.samsung.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: linux-kselftest@vger.kernel.org
Cc: "H . Peter Anvin" <hpa@zytor.com>
Cc: Chris Lameter <cl@linux.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Andrew Hunter <ahh@google.com>
Cc: Michael Kerrisk <mtk.manpages@gmail.com>
Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Paul Turner <pjt@google.com>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ben Maurer <bmaurer@fb.com>
Cc: linux-api@vger.kernel.org
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
---
tools/testing/selftests/rseq/run_param_test.sh | 0
1 file changed, 0 insertions(+), 0 deletions(-)
mode change 100644 => 100755 tools/testing/selftests/rseq/run_param_test.sh
diff --git a/tools/testing/selftests/rseq/run_param_test.sh b/tools/testing/selftests/rseq/run_param_test.sh
old mode 100644
new mode 100755
--
2.11.0
^ permalink raw reply
* Re: [PATCH] mm/madvise: allow MADV_DONTNEED to free memory that is MLOCK_ONFAULT
From: Jason Baron @ 2018-06-15 19:36 UTC (permalink / raw)
To: Michal Hocko, Vlastimil Babka
Cc: akpm, linux-kernel, linux-mm, Joonsoo Kim, Mel Gorman,
Kirill A. Shutemov, linux-api, emunson
In-Reply-To: <20180613071552.GD13364@dhcp22.suse.cz>
On 06/13/2018 03:15 AM, Michal Hocko wrote:
> On Wed 13-06-18 08:32:19, Vlastimil Babka wrote:
>> On 06/12/2018 04:11 PM, Jason Baron wrote:
>>>
>>>
>>> On 06/12/2018 03:46 AM, Michal Hocko wrote:
>>>> On Mon 11-06-18 12:23:58, Jason Baron wrote:
>>>>> On 06/11/2018 11:03 AM, Michal Hocko wrote:
>>>>>> So can we start discussing whether we want to allow MADV_DONTNEED on
>>>>>> mlocked areas and what downsides it might have? Sure it would turn the
>>>>>> strong mlock guarantee to have the whole vma resident but is this
>>>>>> acceptable for something that is an explicit request from the owner of
>>>>>> the memory?
>>>>>>
>>>>>
>>>>> If its being explicity requested by the owner it makes sense to me. I
>>>>> guess there could be a concern about this breaking some userspace that
>>>>> relied on MADV_DONTNEED not freeing locked memory?
>>>>
>>>> Yes, this is always the fear when changing user visible behavior. I can
>>>> imagine that a userspace allocator calling MADV_DONTNEED on free could
>>>> break. The same would apply to MLOCK_ONFAULT/MCL_ONFAULT though. We
>>>> have the new flag much shorter so the probability is smaller but the
>>>> problem is very same. So I _think_ we should treat both the same because
>>>> semantically they are indistinguishable from the MADV_DONTNEED POV. Both
>>>> remove faulted and mlocked pages. Mlock, once applied, should guarantee
>>>> no later major fault and MADV_DONTNEED breaks that obviously.
>>
>> I think more concerning than guaranteeing no later major fault is
>> possible data loss, e.g. replacing data with zero-filled pages.
>
> But MADV_DONTNEED is an explicit call for data loss. Or do I miss your
> point?
>
>> The madvise manpage is also quite specific about not allowing
>> MADV_DONTNEED and MADV_FREE for locked pages.
>
> Yeah, but that seems to describe the state of the art rather than
> explain why.
>
>> So I don't think we should risk changing that for all mlocked pages.
>> Maybe we can risk MCL_ONFAULT, since it's relatively new and has few users?
>
> That is what Jason wanted but I argued that the two are the same from
> MADV_DONTNEED point of view. I do not see how treating them differently
> would be less confusing or error prone. It's new so we can make it
> behave differently is certainly not an argument.
>
>>>> So the more I think about it the more I am worried about this but I am
>>>> more and more convinced that making ONFAULT special is just a wrong way
>>>> around this.
>>>>
>>>
>>> Ok, I share the concern that there is a chance that userspace is relying
>>> on MADV_DONTNEED not free'ing locked memory. In that case, what if we
>>> introduce a MADV_DONTNEED_FORCE, which does everything that
>>> MADV_DONTNEED currently does but in addition will also free mlock areas.
>>> That way there is no concern about breaking something.
>>
>> A new niche case flag? Sad :(
>>
>> BTW I didn't get why we should allow this for MADV_DONTNEED but not
>> MADV_FREE. Can you expand on that?
>
> Well, I wanted to bring this up as well. I guess this would require some
> more hacks to handle the reclaim path correctly because we do rely on
> VM_LOCK at many places for the lazy mlock pages culling.
>
The point of not allowing MADV_FREE on mlock'd pages for me was that
with mlock and even MLOCK_ON_FAULT, one can always can always determine
if a page is present or not (and thus avoid the major fault). Allowing
MADV_FREE on lock'd pages breaks that assumption.
Thanks,
-Jason
^ permalink raw reply
* Re: [PATCH] mm/madvise: allow MADV_DONTNEED to free memory that is MLOCK_ONFAULT
From: Jason Baron @ 2018-06-15 19:28 UTC (permalink / raw)
To: Michal Hocko
Cc: akpm, linux-kernel, linux-mm, Vlastimil Babka, Joonsoo Kim,
Mel Gorman, Kirill A. Shutemov, linux-api, emunson
In-Reply-To: <20180613091355.GI13364@dhcp22.suse.cz>
On 06/13/2018 05:13 AM, Michal Hocko wrote:
> On Tue 12-06-18 10:11:33, Jason Baron wrote:
> [...]
>> Ok, I share the concern that there is a chance that userspace is relying
>> on MADV_DONTNEED not free'ing locked memory. In that case, what if we
>> introduce a MADV_DONTNEED_FORCE, which does everything that
>> MADV_DONTNEED currently does but in addition will also free mlock areas.
>
> What about other types of vmas that are not allowed to MADV_DONTNEED?
> _FORCE suggests that the user of the API know what he is doing so why
> shouldn't we allow unmapping hugetlb pages or special PFNMAPS? Or do we
> want to add MADV_DONTNEED_FORCE_FOR_REAL when somebody comes with
> another usecase?
>
> I agree with Vlastimil that adding new madvise mode for niche case
> sounds like a bad idea so we should better be sure that a new flag has
> a reasonable semantic. Just allow mlocked pages is more of a tweak than
> a proper semantic. So making it force for real requires to analyze what
> that would mean for other vmas which are excluded now.
>
If its a new flag, I agree it makes sense to look at hugetlb and
pfnmaps. pfnmaps might be more work, since it may require callbacks to
do driver specific actions...
I was able to do something very close to the original requirement of
free'ing mlock'd pages, via using a tmpfs mmap that is mlock'd. And then
using fallocate(FALLOC_FL_PUNCH_HOLE) to free the locked pages. I think
the tmpfs is sufficient for my needs, I wonder if there is any other
interest in this feature?
Thanks,
-Jason
^ permalink raw reply
* [patch v26 4/4] Documentation: jtag: Add ABI documentation
From: Oleksandr Shamray @ 2018-06-15 16:45 UTC (permalink / raw)
To: gregkh, arnd
Cc: system-sw-low-level, devicetree, jiri, vadimp, linux-api, openbmc,
linux-kernel, openocd-devel-owner, robh+dt, joel, linux-serial,
Oleksandr Shamray, tklauser, mchehab, davem, linux-arm-kernel
In-Reply-To: <1529081123-30873-1-git-send-email-oleksandrs@mellanox.com>
Added document that describe the ABI for JTAG class drivrer
Signed-off-by: Oleksandr Shamray <oleksandrs@mellanox.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
---
v25->v26
Comments pointed by Randy Dunlap <rdunlap@infradead.org>
- fix spell in ABI doccumentation
v24->v25
Comments pointed by Greg KH <gregkh@linuxfoundation.org>
- Fixed documentation according to new open() behavior
v23->v24
v22->v23
Comments pointed by Randy Dunlap <rdunlap@infradead.org>
- fix spell in ABI doccumentation
v21->v22
Comments pointed by Randy Dunlap <rdunlap@infradead.org>
- fix spell in ABI doccumentation
v20->v21
Comments pointed by Randy Dunlap <rdunlap@infradead.org>
- Fix JTAG dirver help in Kconfig
v19->v20
Comments pointed by Randy Dunlap <rdunlap@infradead.org>
- Fix JTAG doccumentation
v18->v19
Pavel Machek <pavel@ucw.cz>
- Added JTAG doccumentation to Documentation/jtag
v17->v18
v16->v17
v15->v16
v14->v15
v13->v14
v12->v13
v11->v12 Tobias Klauser <tklauser@distanz.ch>
Comments pointed by
- rename /Documentation/ABI/testing/jatg-dev -> jtag-dev
- Typo: s/interfase/interface
v10->v11
v9->v10
Fixes added by Oleksandr:
- change jtag-cdev to jtag-dev in documentation
- update KernelVersion and Date in jtag-dev documentation;
v8->v9
v7->v8
v6->v7
Comments pointed by Pavel Machek <pavel@ucw.cz>
- Added jtag-cdev documentation to Documentation/ABI/testing folder
---
Documentation/ABI/testing/gpio-cdev | 1 -
Documentation/ABI/testing/jtag-dev | 23 +++++++
Documentation/jtag/overview | 27 +++++++++
Documentation/jtag/transactions | 109 +++++++++++++++++++++++++++++++++++
MAINTAINERS | 1 +
5 files changed, 160 insertions(+), 1 deletions(-)
create mode 100644 Documentation/ABI/testing/jtag-dev
create mode 100644 Documentation/jtag/overview
create mode 100644 Documentation/jtag/transactions
diff --git a/Documentation/ABI/testing/gpio-cdev b/Documentation/ABI/testing/gpio-cdev
index 7b265fb..c09d589 100644
--- a/Documentation/ABI/testing/gpio-cdev
+++ b/Documentation/ABI/testing/gpio-cdev
@@ -12,7 +12,6 @@ Description:
The following file operations are supported:
open(2)
- Currently the only useful flags are O_RDWR.
ioctl(2)
Initiate various actions.
diff --git a/Documentation/ABI/testing/jtag-dev b/Documentation/ABI/testing/jtag-dev
new file mode 100644
index 0000000..08249fe
--- /dev/null
+++ b/Documentation/ABI/testing/jtag-dev
@@ -0,0 +1,23 @@
+What: /dev/jtag[0-9]+
+Date: May 2018
+KernelVersion: 4.18
+Contact: oleksandrs@mellanox.com
+Description:
+ The misc device files /dev/jtag* are the interface
+ between JTAG master interface and userspace.
+
+ The ioctl(2)-based ABI is defined and documented in
+ [include/uapi]<linux/jtag.h>.
+
+ The following file operations are supported:
+
+ open(2)
+ Opens and allocates file descriptor.
+
+ ioctl(2)
+ Initiate various actions.
+ See the inline documentation in [include/uapi]<linux/jtag.h>
+ for descriptions of all ioctls.
+
+Users:
+ userspace tools which wants to access to JTAG bus
diff --git a/Documentation/jtag/overview b/Documentation/jtag/overview
new file mode 100644
index 0000000..3c39052
--- /dev/null
+++ b/Documentation/jtag/overview
@@ -0,0 +1,27 @@
+Linux kernel JTAG support
+=========================
+
+JTAG is an industry standard for verifying hardware. JTAG provides access to
+many logic signals of a complex integrated circuit, including the device pins.
+
+A JTAG interface is a special interface added to a chip.
+Depending on the version of JTAG, two, four, or five pins are added.
+
+The connector pins are:
+ TDI (Test Data In)
+ TDO (Test Data Out)
+ TCK (Test Clock)
+ TMS (Test Mode Select)
+ TRST (Test Reset) optional
+
+JTAG interface is designed to have two parts - basic core driver and
+hardware specific driver. The basic driver introduces a general interface
+which is not dependent of specific hardware. It provides communication
+between user space and hardware specific driver.
+Each JTAG device is represented as a char device from (jtag0, jtag1, ...).
+Access to a JTAG device is performed through IOCTL calls.
+
+Call flow example:
+User: open -> /dev/jatgX
+User: ioctl -> /dev/jtagX -> JTAG core driver -> JTAG hardware specific driver
+User: close -> /dev/jatgX
diff --git a/Documentation/jtag/transactions b/Documentation/jtag/transactions
new file mode 100644
index 0000000..e86fb73
--- /dev/null
+++ b/Documentation/jtag/transactions
@@ -0,0 +1,109 @@
+The JTAG API
+=============
+
+JTAG master devices can be accessed through a character misc-device.
+Each JTAG master interface can be accessed by using /dev/jtagN.
+
+JTAG system calls set:
+- SIR (Scan Instruction Register, IEEE 1149.1 Instruction Register scan);
+- SDR (Scan Data Register, IEEE 1149.1 Data Register scan);
+- RUNTEST (Forces the IEEE 1149.1 bus to a run state for a specified
+number of clocks.
+
+open(), close()
+-------
+open() opens JTAG device.
+
+Open/Close device:
+- jtag_fd = open("/dev/jtag0", O_RDWR);
+- close(jtag_fd);
+
+ioctl()
+-------
+All access operations to JTAG devices are performed through ioctl interface.
+The IOCTL interface supports these requests:
+ JTAG_IOCRUNTEST - Force JTAG state machine to RUN_TEST/IDLE state
+ JTAG_SIOCFREQ - Set JTAG TCK frequency
+ JTAG_GIOCFREQ - Get JTAG TCK frequency
+ JTAG_IOCXFER - send JTAG data Xfer
+ JTAG_GIOCSTATUS - get current JTAG TAP status
+ JTAG_SIOCMODE - set JTAG mode flags.
+
+JTAG_SIOCFREQ, JTAG_GIOCFREQ
+------
+Set/Get JTAG clock speed:
+
+ unsigned int jtag_fd;
+ ioctl(jtag_fd, JTAG_SIOCFREQ, &frq);
+ ioctl(jtag_fd, JTAG_GIOCFREQ, &frq);
+
+JTAG_IOCRUNTEST
+------
+Force JTAG state machine to RUN_TEST/IDLE state
+
+struct jtag_run_test_idle {
+ __u8 reset;
+ __u8 endstate;
+ __u8 tck;
+};
+
+reset:
+ JTAG_NO_RESET - run IDLE/PAUSE from current state
+ JTAG_FORCE_RESET - go through TEST_LOGIC/RESET state before IDLE/PAUSE
+endstate: completion flag
+tck: clock counter
+
+Example:
+ struct jtag_run_test_idle runtest;
+
+ runtest.endstate = JTAG_STATE_IDLE;
+ runtest.reset = 0;
+ runtest.tck = data_p->tck;
+ usleep(25 * 1000);
+ ioctl(jtag_fd, JTAG_IOCRUNTEST, &runtest);
+
+JTAG_IOCXFER
+------
+Send SDR/SIR transaction
+
+struct jtag_xfer {
+ __u8 type;
+ __u8 direction;
+ __u8 endstate;
+ __u8 padding;
+ __u32 length;
+ __u64 tdio;
+};
+
+type: transfer type - JTAG_SIR_XFER/JTAG_SDR_XFER
+direction: xfer direction - JTAG_SIR_XFER/JTAG_SDR_XFER
+length: xfer data length in bits
+tdio : xfer data array
+endstate: xfer end state after transaction finish
+ can be: JTAG_STATE_IDLE/JTAG_STATE_PAUSEIR/JTAG_STATE_PAUSEDR
+
+Example:
+ struct jtag_xfer xfer;
+ static char buf[64];
+ static unsigned int buf_len = 0;
+ [...]
+ xfer.type = JTAG_SDR_XFER;
+ xfer.tdio = (__u64)buf;
+ xfer.length = buf_len;
+ xfer.endstate = JTAG_STATE_IDLE;
+
+ if (is_read)
+ xfer.direction = JTAG_READ_XFER;
+ else
+ xfer.direction = JTAG_WRITE_XFER;
+
+ ioctl(jtag_fd, JTAG_IOCXFER, &xfer);
+
+JTAG_SIOCMODE
+------
+If hardware driver can support different running modes you can change it.
+
+Example:
+ unsigned int mode;
+ mode = JTAG_XFER_HW_MODE;
+ ioctl(jtag_fd, JTAG_SIOCMODE, &mode);
diff --git a/MAINTAINERS b/MAINTAINERS
index a5e5f75..e067eda 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7618,6 +7618,7 @@ F: include/linux/jtag.h
F: include/uapi/linux/jtag.h
F: drivers/jtag/
F: Documentation/devicetree/bindings/jtag/
+F: Documentation/ABI/testing/jtag-dev
K10TEMP HARDWARE MONITORING DRIVER
M: Clemens Ladisch <clemens@ladisch.de>
--
1.7.1
^ permalink raw reply related
* [patch v26 3/4] Documentation: jtag: Add bindings for Aspeed SoC 24xx and 25xx families JTAG master driver
From: Oleksandr Shamray @ 2018-06-15 16:45 UTC (permalink / raw)
To: gregkh, arnd
Cc: system-sw-low-level, devicetree, jiri, vadimp, linux-api, openbmc,
linux-kernel, openocd-devel-owner, robh+dt, joel, linux-serial,
Oleksandr Shamray, tklauser, mchehab, davem, linux-arm-kernel
In-Reply-To: <1529081123-30873-1-git-send-email-oleksandrs@mellanox.com>
It has been tested on Mellanox system with BMC equipped with
Aspeed 2520 SoC for programming CPLD devices.
Signed-off-by: Oleksandr Shamray <oleksandrs@mellanox.com>
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Acked-by: Rob Herring <robh@kernel.org>
---
v21->v22
v20->v21
v19->v20
v18->v19
v17->v18
v16->v17
v15->v16
Comments pointed by Joel Stanley <joel.stan@gmail.com>
- change clocks = <&clk_apb> to proper clocks = <&syscon ASPEED_CLK_APB>
- add reset descriptions in bndings file
v14->v15
v13->v14
v12->v13
v11->v12
v10->v11
v9->v10
v8->v9
v7->v8
Comments pointed by pointed by Joel Stanley <joel.stan@gmail.com>
- Change compatible string to ast2400 and ast2000
V6->v7
Comments pointed by Tobias Klauser <tklauser@distanz.ch>
- Fix spell "Doccumentation" -> "Documentation"
v5->v6
Comments pointed by Tobias Klauser <tklauser@distanz.ch>
- Small nit: s/documentation/Documentation/
v4->v5
V3->v4
Comments pointed by Rob Herring <robh@kernel.org>
- delete unnecessary "status" and "reg-shift" descriptions in
bndings file
v2->v3
Comments pointed by Rob Herring <robh@kernel.org>
- split Aspeed jtag driver and binding to sepatrate patches
- delete unnecessary "status" and "reg-shift" descriptions in
bndings file
---
.../devicetree/bindings/jtag/aspeed-jtag.txt | 22 ++++++++++++++++++++
MAINTAINERS | 1 +
2 files changed, 23 insertions(+), 0 deletions(-)
create mode 100644 Documentation/devicetree/bindings/jtag/aspeed-jtag.txt
diff --git a/Documentation/devicetree/bindings/jtag/aspeed-jtag.txt b/Documentation/devicetree/bindings/jtag/aspeed-jtag.txt
new file mode 100644
index 0000000..7c36eb6
--- /dev/null
+++ b/Documentation/devicetree/bindings/jtag/aspeed-jtag.txt
@@ -0,0 +1,22 @@
+Aspeed JTAG driver for ast2400 and ast2500 SoC
+
+Required properties:
+- compatible: Should be one of
+ - "aspeed,ast2400-jtag"
+ - "aspeed,ast2500-jtag"
+- reg contains the offset and length of the JTAG memory
+ region
+- clocks root clock of bus, should reference the APB
+ clock in the second cell
+- resets phandle to reset controller with the reset number in
+ the second cell
+- interrupts should contain JTAG controller interrupt
+
+Example:
+jtag: jtag@1e6e4000 {
+ compatible = "aspeed,ast2500-jtag";
+ reg = <0x1e6e4000 0x1c>;
+ clocks = <&syscon ASPEED_CLK_APB>;
+ resets = <&syscon ASPEED_RESET_JTAG_MASTER>;
+ interrupts = <43>;
+};
diff --git a/MAINTAINERS b/MAINTAINERS
index e7b8b2c..a5e5f75 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7617,6 +7617,7 @@ S: Maintained
F: include/linux/jtag.h
F: include/uapi/linux/jtag.h
F: drivers/jtag/
+F: Documentation/devicetree/bindings/jtag/
K10TEMP HARDWARE MONITORING DRIVER
M: Clemens Ladisch <clemens@ladisch.de>
--
1.7.1
^ permalink raw reply related
* [patch v26 2/4] drivers: jtag: Add Aspeed SoC 24xx and 25xx families JTAG master driver
From: Oleksandr Shamray @ 2018-06-15 16:45 UTC (permalink / raw)
To: gregkh, arnd
Cc: linux-kernel, linux-arm-kernel, devicetree, openbmc, joel, jiri,
tklauser, linux-serial, vadimp, system-sw-low-level, robh+dt,
openocd-devel-owner, linux-api, davem, mchehab, Oleksandr Shamray
In-Reply-To: <1529081123-30873-1-git-send-email-oleksandrs@mellanox.com>
Driver adds support of Aspeed 2500/2400 series SOC JTAG master controller.
Driver implements the following jtag ops:
- freq_get;
- freq_set;
- status_get;
- idle;
- xfer;
It has been tested on Mellanox system with BMC equipped with
Aspeed 2520 SoC for programming CPLD devices.
Signed-off-by: Oleksandr Shamray <oleksandrs@mellanox.com>
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Philippe Ombredanne <pombredanne@nexb.com>
Acked-by: Joel Stanley <joel@jms.id.au>
---
v24->v25
Comments pointed by Greg KH <gregkh@linuxfoundation.org>
- reduced debug printouts
v23->v24
v22->v23
v21->v22
Comments pointed by Andy Shevchenko <andy.shevchenko@gmail.com>
- rearrange ASPEED register defines
- simplified JTAG divider calculation formula
- change delay function in bit-bang operation
- add helper functions for TAP states switching
- remove unnecessary comments
- remove redundant debug messages
- make dines for repetative register bit sets
- fixed indentation
- change checks from negative to positive
- add error check for clk_prepare_enable
- rework driver alloc/register to devm_ variant
- Increasing line length up to 85 in order to improve readability
v20->v21
v19->v20
Notifications from kbuild test robot <lkp@intel.com>
- add static declaration to 'aspeed_jtag_init' and
'aspeed_jtag_deinit' functions
v18->v19
v17->v18
v16->v17
v15->v16
Comments pointed by Joel Stanley <joel.stan@gmail.com>
- Add reset_control on Jtag init/deinit
v14->v15
Comments pointed by Joel Stanley <joel.stan@gmail.com>
- Add ARCH_ASPEED || COMPILE_TEST to Kconfig
- remove unused offset variable
- remove "aspeed_jtag" from dev_err and dev_dbg messages
- change clk_prepare_enable initialisation order
v13->v14
Comments pointed by Philippe Ombredanne <pombredanne@nexb.com>
- Change style of head block comment from /**/ to //
v12->v13
Comments pointed by Philippe Ombredanne <pombredanne@nexb.com>
- Change jtag-aspeed.c licence type to
SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note
and reorder line with license- add reset descriptions in bndings file
in description
Comments pointed by Kun Yi <kunyi@google.com>
- Changed capability check for aspeed,ast2400-jtag/ast200-jtag
v11->v12
Comments pointed by Chip Bilbrey <chip@bilbrey.org>
- Remove access mode from xfer and idle transactions
- Add new ioctl JTAG_SIOCMODE for set hw mode
v10->v11
v9->v10
V8->v9
Comments pointed by Arnd Bergmann <arnd@arndb.de>
- add *data parameter to xfer function prototype
v7->v8
Comments pointed by Joel Stanley <joel.stan@gmail.com>
- aspeed_jtag_init replace goto to return;
- change input variables type from __u32 to u32
in functios freq_get, freq_set, status_get
- change sm_ variables type from char to u8
- in jatg_init add disable clocks on error case
- remove release_mem_region on error case
- remove devm_free_irq on jtag_deinit
- Fix misspelling Disabe/Disable
- Change compatible string to ast2400 and ast2000
v6->v7
Notifications from kbuild test robot <lkp@intel.com>
- Add include <linux/types.h> to jtag-asapeed.c
v5->v6
v4->v5
Comments pointed by Arnd Bergmann <arnd@arndb.de>
- Added HAS_IOMEM dependence in Kconfig to avoid
"undefined reference to `devm_ioremap_resource'" error,
because in some arch this not supported
v3->v4
Comments pointed by Arnd Bergmann <arnd@arndb.de>
- change transaction pointer tdio type to __u64
- change internal status type from enum to __u32
v2->v3
v1->v2
Comments pointed by Greg KH <gregkh@linuxfoundation.org>
- change license type from GPLv2/BSD to GPLv2
Comments pointed by Neil Armstrong <narmstrong@baylibre.com>
- Add clk_prepare_enable/clk_disable_unprepare in clock init/deinit
- Change .compatible to soc-specific compatible names
aspeed,aspeed4000-jtag/aspeed5000-jtag
- Added dt-bindings
Comments pointed by Arnd Bergmann <arnd@arndb.de>
- Reorder functions and removed the forward declarations
- Add static const qualifier to state machine states transitions
- Change .compatible to soc-specific compatible names
aspeed,aspeed4000-jtag/aspeed5000-jtag
- Add dt-bindings
Comments pointed by Randy Dunlap <rdunlap@infradead.org>
- Change module name jtag-aspeed in description in Kconfig
Comments pointed by kbuild test robot <lkp@intel.com>
- Remove invalid include <asm/mach-types.h>
- add resource_size instead of calculation
---
drivers/jtag/Kconfig | 14 +
drivers/jtag/Makefile | 1 +
drivers/jtag/jtag-aspeed.c | 756 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 771 insertions(+), 0 deletions(-)
create mode 100644 drivers/jtag/jtag-aspeed.c
diff --git a/drivers/jtag/Kconfig b/drivers/jtag/Kconfig
index 47771fc..0cc163f 100644
--- a/drivers/jtag/Kconfig
+++ b/drivers/jtag/Kconfig
@@ -15,3 +15,17 @@ menuconfig JTAG
To compile this driver as a module, choose M here: the module will
be called jtag.
+
+menuconfig JTAG_ASPEED
+ tristate "Aspeed SoC JTAG controller support"
+ depends on JTAG && HAS_IOMEM
+ depends on ARCH_ASPEED || COMPILE_TEST
+ help
+ This provides a support for Aspeed JTAG device, equipped on
+ Aspeed SoC 24xx and 25xx families. Drivers allows programming
+ of hardware devices, connected to SoC through the JTAG interface.
+
+ If you want this support, you should say Y here.
+
+ To compile this driver as a module, choose M here: the module will
+ be called jtag-aspeed.
diff --git a/drivers/jtag/Makefile b/drivers/jtag/Makefile
index af37493..04a855e 100644
--- a/drivers/jtag/Makefile
+++ b/drivers/jtag/Makefile
@@ -1 +1,2 @@
obj-$(CONFIG_JTAG) += jtag.o
+obj-$(CONFIG_JTAG_ASPEED) += jtag-aspeed.o
diff --git a/drivers/jtag/jtag-aspeed.c b/drivers/jtag/jtag-aspeed.c
new file mode 100644
index 0000000..15ae188
--- /dev/null
+++ b/drivers/jtag/jtag-aspeed.c
@@ -0,0 +1,756 @@
+// SPDX-License-Identifier: GPL-2.0
+// drivers/jtag/aspeed-jtag.c
+//
+// Copyright (c) 2018 Mellanox Technologies. All rights reserved.
+// Copyright (c) 2018 Oleksandr Shamray <oleksandrs@mellanox.com>
+
+#include <linux/clk.h>
+#include <linux/device.h>
+#include <linux/interrupt.h>
+#include <linux/jtag.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/reset.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/delay.h>
+#include <uapi/linux/jtag.h>
+
+#define ASPEED_JTAG_DATA 0x00
+#define ASPEED_JTAG_INST 0x04
+#define ASPEED_JTAG_CTRL 0x08
+#define ASPEED_JTAG_ISR 0x0C
+#define ASPEED_JTAG_SW 0x10
+#define ASPEED_JTAG_TCK 0x14
+#define ASPEED_JTAG_EC 0x18
+
+#define ASPEED_JTAG_DATA_MSB 0x01
+#define ASPEED_JTAG_DATA_CHUNK_SIZE 0x20
+
+/* ASPEED_JTAG_CTRL: Engine Control */
+#define ASPEED_JTAG_CTL_ENG_EN BIT(31)
+#define ASPEED_JTAG_CTL_ENG_OUT_EN BIT(30)
+#define ASPEED_JTAG_CTL_FORCE_TMS BIT(29)
+#define ASPEED_JTAG_CTL_INST_LEN(x) ((x) << 20)
+#define ASPEED_JTAG_CTL_LASPEED_INST BIT(17)
+#define ASPEED_JTAG_CTL_INST_EN BIT(16)
+#define ASPEED_JTAG_CTL_DR_UPDATE BIT(10)
+#define ASPEED_JTAG_CTL_DATA_LEN(x) ((x) << 4)
+#define ASPEED_JTAG_CTL_LASPEED_DATA BIT(1)
+#define ASPEED_JTAG_CTL_DATA_EN BIT(0)
+
+/* ASPEED_JTAG_ISR : Interrupt status and enable */
+#define ASPEED_JTAG_ISR_INST_PAUSE BIT(19)
+#define ASPEED_JTAG_ISR_INST_COMPLETE BIT(18)
+#define ASPEED_JTAG_ISR_DATA_PAUSE BIT(17)
+#define ASPEED_JTAG_ISR_DATA_COMPLETE BIT(16)
+#define ASPEED_JTAG_ISR_INST_PAUSE_EN BIT(3)
+#define ASPEED_JTAG_ISR_INST_COMPLETE_EN BIT(2)
+#define ASPEED_JTAG_ISR_DATA_PAUSE_EN BIT(1)
+#define ASPEED_JTAG_ISR_DATA_COMPLETE_EN BIT(0)
+#define ASPEED_JTAG_ISR_INT_EN_MASK GENMASK(3, 0)
+#define ASPEED_JTAG_ISR_INT_MASK GENMASK(19, 16)
+
+/* ASPEED_JTAG_SW : Software Mode and Status */
+#define ASPEED_JTAG_SW_MODE_EN BIT(19)
+#define ASPEED_JTAG_SW_MODE_TCK BIT(18)
+#define ASPEED_JTAG_SW_MODE_TMS BIT(17)
+#define ASPEED_JTAG_SW_MODE_TDIO BIT(16)
+
+/* ASPEED_JTAG_TCK : TCK Control */
+#define ASPEED_JTAG_TCK_DIVISOR_MASK GENMASK(10, 0)
+#define ASPEED_JTAG_TCK_GET_DIV(x) ((x) & ASPEED_JTAG_TCK_DIVISOR_MASK)
+
+/* ASPEED_JTAG_EC : Controller set for go to IDLE */
+#define ASPEED_JTAG_EC_GO_IDLE BIT(0)
+
+#define ASPEED_JTAG_IOUT_LEN(len) \
+ (ASPEED_JTAG_CTL_ENG_EN | \
+ ASPEED_JTAG_CTL_ENG_OUT_EN | \
+ ASPEED_JTAG_CTL_INST_LEN(len))
+
+#define ASPEED_JTAG_DOUT_LEN(len) \
+ (ASPEED_JTAG_CTL_ENG_EN | \
+ ASPEED_JTAG_CTL_ENG_OUT_EN | \
+ ASPEED_JTAG_CTL_DATA_LEN(len))
+
+#define ASPEED_JTAG_SW_TDIO (ASPEED_JTAG_SW_MODE_EN | ASPEED_JTAG_SW_MODE_TDIO)
+
+#define ASPEED_JTAG_GET_TDI(direction, byte) \
+ ((direction == JTAG_READ_XFER) ? UINT_MAX : byte)
+
+#define ASPEED_JTAG_TCK_WAIT 10
+#define ASPEED_JTAG_RESET_CNTR 10
+
+#define ASPEED_JTAG_NAME "jtag-aspeed"
+
+struct aspeed_jtag {
+ void __iomem *reg_base;
+ struct device *dev;
+ struct clk *pclk;
+ enum jtag_endstate status;
+ int irq;
+ struct reset_control *rst;
+ u32 flag;
+ wait_queue_head_t jtag_wq;
+ u32 mode;
+};
+
+static char *end_status_str[] = {"idle", "ir pause", "drpause"};
+
+static u32 aspeed_jtag_read(struct aspeed_jtag *aspeed_jtag, u32 reg)
+{
+ return readl(aspeed_jtag->reg_base + reg);
+}
+
+static void
+aspeed_jtag_write(struct aspeed_jtag *aspeed_jtag, u32 val, u32 reg)
+{
+ writel(val, aspeed_jtag->reg_base + reg);
+}
+
+static int aspeed_jtag_freq_set(struct jtag *jtag, u32 freq)
+{
+ struct aspeed_jtag *aspeed_jtag = jtag_priv(jtag);
+ unsigned long apb_frq;
+ u32 tck_val;
+ u16 div;
+
+ apb_frq = clk_get_rate(aspeed_jtag->pclk);
+ if (!apb_frq)
+ return -ENOTSUPP;
+
+ div = (apb_frq - 1) / freq;
+ tck_val = aspeed_jtag_read(aspeed_jtag, ASPEED_JTAG_TCK);
+ aspeed_jtag_write(aspeed_jtag,
+ (tck_val & ASPEED_JTAG_TCK_DIVISOR_MASK) | div,
+ ASPEED_JTAG_TCK);
+ return 0;
+}
+
+static int aspeed_jtag_freq_get(struct jtag *jtag, u32 *frq)
+{
+ struct aspeed_jtag *aspeed_jtag = jtag_priv(jtag);
+ u32 pclk;
+ u32 tck;
+
+ pclk = clk_get_rate(aspeed_jtag->pclk);
+ tck = aspeed_jtag_read(aspeed_jtag, ASPEED_JTAG_TCK);
+ *frq = pclk / (ASPEED_JTAG_TCK_GET_DIV(tck) + 1);
+
+ return 0;
+}
+
+static int aspeed_jtag_mode_set(struct jtag *jtag, u32 mode)
+{
+ struct aspeed_jtag *aspeed_jtag = jtag_priv(jtag);
+
+ aspeed_jtag->mode = mode;
+ return 0;
+}
+
+static void aspeed_jtag_sw_delay(struct aspeed_jtag *aspeed_jtag, int cnt)
+{
+ int i;
+
+ for (i = 0; i < cnt; i++)
+ aspeed_jtag_read(aspeed_jtag, ASPEED_JTAG_SW);
+}
+
+static char aspeed_jtag_tck_cycle(struct aspeed_jtag *aspeed_jtag,
+ u8 tms, u8 tdi)
+{
+ char tdo = 0;
+
+ /* TCK = 0 */
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_SW_MODE_EN |
+ (tms * ASPEED_JTAG_SW_MODE_TMS) |
+ (tdi * ASPEED_JTAG_SW_MODE_TDIO), ASPEED_JTAG_SW);
+
+ ndelay(ASPEED_JTAG_TCK_WAIT);
+ aspeed_jtag_sw_delay(aspeed_jtag, ASPEED_JTAG_TCK_WAIT);
+
+ /* TCK = 1 */
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_SW_MODE_EN |
+ ASPEED_JTAG_SW_MODE_TCK |
+ (tms * ASPEED_JTAG_SW_MODE_TMS) |
+ (tdi * ASPEED_JTAG_SW_MODE_TDIO), ASPEED_JTAG_SW);
+
+ if (aspeed_jtag_read(aspeed_jtag, ASPEED_JTAG_SW) &
+ ASPEED_JTAG_SW_MODE_TDIO)
+ tdo = 1;
+
+ aspeed_jtag_sw_delay(aspeed_jtag, ASPEED_JTAG_TCK_WAIT);
+
+ /* TCK = 0 */
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_SW_MODE_EN |
+ (tms * ASPEED_JTAG_SW_MODE_TMS) |
+ (tdi * ASPEED_JTAG_SW_MODE_TDIO), ASPEED_JTAG_SW);
+ return tdo;
+}
+
+static void aspeed_jtag_wait_instruction_pause(struct aspeed_jtag *aspeed_jtag)
+{
+ wait_event_interruptible(aspeed_jtag->jtag_wq,
+ aspeed_jtag->flag & ASPEED_JTAG_ISR_INST_PAUSE);
+ aspeed_jtag->flag &= ~ASPEED_JTAG_ISR_INST_PAUSE;
+}
+
+static void
+aspeed_jtag_wait_instruction_complete(struct aspeed_jtag *aspeed_jtag)
+{
+ wait_event_interruptible(aspeed_jtag->jtag_wq,
+ aspeed_jtag->flag & ASPEED_JTAG_ISR_INST_COMPLETE);
+ aspeed_jtag->flag &= ~ASPEED_JTAG_ISR_INST_COMPLETE;
+}
+
+static void
+aspeed_jtag_wait_data_pause_complete(struct aspeed_jtag *aspeed_jtag)
+{
+ wait_event_interruptible(aspeed_jtag->jtag_wq,
+ aspeed_jtag->flag & ASPEED_JTAG_ISR_DATA_PAUSE);
+ aspeed_jtag->flag &= ~ASPEED_JTAG_ISR_DATA_PAUSE;
+}
+
+static void aspeed_jtag_wait_data_complete(struct aspeed_jtag *aspeed_jtag)
+{
+ wait_event_interruptible(aspeed_jtag->jtag_wq,
+ aspeed_jtag->flag & ASPEED_JTAG_ISR_DATA_COMPLETE);
+ aspeed_jtag->flag &= ~ASPEED_JTAG_ISR_DATA_COMPLETE;
+}
+
+static void aspeed_jtag_sm_cycle(struct aspeed_jtag *aspeed_jtag,
+ const u8 *tms, int len)
+{
+ int i;
+
+ for (i = 0; i < len; i++)
+ aspeed_jtag_tck_cycle(aspeed_jtag, tms[i], 0);
+}
+
+static void aspeed_jtag_run_idle(struct aspeed_jtag *aspeed_jtag,
+ struct jtag_run_test_idle *runtest)
+{
+ static const u8 sm_idle_irpause[] = {1, 1, 0, 1, 0};
+ static const u8 sm_idle_drpause[] = {1, 0, 1, 0};
+
+ switch (runtest->endstate) {
+ case JTAG_STATE_PAUSEIR:
+ /* ->DRSCan->IRSCan->IRCap->IRExit1->PauseIR */
+ aspeed_jtag_sm_cycle(aspeed_jtag, sm_idle_irpause,
+ sizeof(sm_idle_irpause));
+ aspeed_jtag->status = JTAG_STATE_PAUSEIR;
+ break;
+ case JTAG_STATE_PAUSEDR:
+ /* ->DRSCan->DRCap->DRExit1->PauseDR */
+ aspeed_jtag_sm_cycle(aspeed_jtag, sm_idle_drpause,
+ sizeof(sm_idle_drpause));
+
+ aspeed_jtag->status = JTAG_STATE_PAUSEDR;
+ break;
+ case JTAG_STATE_IDLE:
+ /* IDLE */
+ aspeed_jtag_tck_cycle(aspeed_jtag, 0, 0);
+ aspeed_jtag->status = JTAG_STATE_IDLE;
+ break;
+ default:
+ break;
+ }
+}
+
+static void aspeed_jtag_run_pause(struct aspeed_jtag *aspeed_jtag,
+ struct jtag_run_test_idle *runtest)
+{
+ static const u8 sm_pause_irpause[] = {1, 1, 1, 1, 0, 1, 0};
+ static const u8 sm_pause_drpause[] = {1, 1, 1, 0, 1, 0};
+ static const u8 sm_pause_idle[] = {1, 1, 0};
+
+ /* From IR/DR Pa.use */
+ switch (runtest->endstate) {
+ case JTAG_STATE_PAUSEIR:
+ /*
+ * to Exit2 IR/DR->Updt IR/DR->DRSCan->IRSCan->IRCap->
+ * IRExit1->PauseIR
+ */
+ aspeed_jtag_sm_cycle(aspeed_jtag, sm_pause_irpause,
+ sizeof(sm_pause_irpause));
+
+ aspeed_jtag->status = JTAG_STATE_PAUSEIR;
+ break;
+ case JTAG_STATE_PAUSEDR:
+ /*
+ * to Exit2 IR/DR->Updt IR/DR->DRSCan->DRCap->
+ * DRExit1->PauseDR
+ */
+ aspeed_jtag_sm_cycle(aspeed_jtag, sm_pause_drpause,
+ sizeof(sm_pause_drpause));
+ aspeed_jtag->status = JTAG_STATE_PAUSEDR;
+ break;
+ case JTAG_STATE_IDLE:
+ /* to Exit2 IR/DR->Updt IR/DR->IDLE */
+ aspeed_jtag_sm_cycle(aspeed_jtag, sm_pause_idle,
+ sizeof(sm_pause_idle));
+ aspeed_jtag->status = JTAG_STATE_IDLE;
+ break;
+ default:
+ break;
+ }
+}
+
+static void aspeed_jtag_run_test_idle_sw(struct aspeed_jtag *aspeed_jtag,
+ struct jtag_run_test_idle *runtest)
+{
+ int i;
+
+ /* SW mode from idle/pause-> to pause/idle */
+ if (runtest->reset) {
+ for (i = 0; i < ASPEED_JTAG_RESET_CNTR; i++)
+ aspeed_jtag_tck_cycle(aspeed_jtag, 1, 0);
+ }
+
+ switch (aspeed_jtag->status) {
+ case JTAG_STATE_IDLE:
+ aspeed_jtag_run_idle(aspeed_jtag, runtest);
+ break;
+
+ case JTAG_STATE_PAUSEIR:
+ /* Fall-through */
+ case JTAG_STATE_PAUSEDR:
+ aspeed_jtag_run_pause(aspeed_jtag, runtest);
+ break;
+
+ default:
+ dev_err(aspeed_jtag->dev, "aspeed_jtag_run_test_idle error\n");
+ break;
+ }
+
+ /* Stay on IDLE for at least TCK cycle */
+ for (i = 0; i < runtest->tck; i++)
+ aspeed_jtag_tck_cycle(aspeed_jtag, 0, 0);
+}
+
+static int aspeed_jtag_idle(struct jtag *jtag,
+ struct jtag_run_test_idle *runtest)
+{
+ struct aspeed_jtag *aspeed_jtag = jtag_priv(jtag);
+
+ dev_dbg(aspeed_jtag->dev, "runtest, state:%s\n",
+ end_status_str[runtest->endstate]);
+
+ if (!(aspeed_jtag->mode & JTAG_XFER_HW_MODE)) {
+ aspeed_jtag_run_test_idle_sw(aspeed_jtag, runtest);
+ return 0;
+ }
+
+ /* Disable sw mode */
+ aspeed_jtag_write(aspeed_jtag, 0, ASPEED_JTAG_SW);
+ /* x TMS high + 1 TMS low */
+ if (runtest->reset)
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_CTL_ENG_EN |
+ ASPEED_JTAG_CTL_ENG_OUT_EN |
+ ASPEED_JTAG_CTL_FORCE_TMS, ASPEED_JTAG_CTRL);
+ else
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_EC_GO_IDLE,
+ ASPEED_JTAG_EC);
+
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_SW_TDIO, ASPEED_JTAG_SW);
+
+ aspeed_jtag->status = JTAG_STATE_IDLE;
+ return 0;
+}
+
+static void aspeed_jtag_xfer_sw(struct aspeed_jtag *aspeed_jtag,
+ struct jtag_xfer *xfer, u32 *data)
+{
+ static const u8 sm_update_shiftir[] = { 1, 1, 0, 0 };
+ static const u8 sm_update_shiftdr[] = { 1, 0, 0 };
+ static const u8 sm_pause_idle[] = { 1, 1, 0 };
+ static const u8 sm_pause_update[] = { 1, 1 };
+ unsigned long remain_xfer = xfer->length;
+ unsigned long shift_bits = 0;
+ unsigned long index = 0;
+ unsigned long tdi;
+ char tdo;
+
+ if (aspeed_jtag->status != JTAG_STATE_IDLE) {
+ /*IR/DR Pause->Exit2 IR / DR->Update IR /DR */
+ aspeed_jtag_sm_cycle(aspeed_jtag, sm_pause_update,
+ sizeof(sm_pause_update));
+ }
+
+ if (xfer->type == JTAG_SIR_XFER)
+ /* ->IRSCan->CapIR->ShiftIR */
+ aspeed_jtag_sm_cycle(aspeed_jtag, sm_update_shiftir,
+ sizeof(sm_update_shiftir));
+ else
+ /* ->DRScan->DRCap->DRShift */
+ aspeed_jtag_sm_cycle(aspeed_jtag, sm_update_shiftdr,
+ sizeof(sm_update_shiftdr));
+
+ tdi = ASPEED_JTAG_GET_TDI(xfer->direction, data[index]);
+
+ while (remain_xfer > 1) {
+ tdo = aspeed_jtag_tck_cycle(aspeed_jtag, 0,
+ tdi & ASPEED_JTAG_DATA_MSB);
+ data[index] |= tdo << (shift_bits %
+ ASPEED_JTAG_DATA_CHUNK_SIZE);
+
+ tdi >>= 1;
+ shift_bits++;
+ remain_xfer--;
+
+ if (shift_bits % ASPEED_JTAG_DATA_CHUNK_SIZE == 0) {
+ tdo = 0;
+ index++;
+
+ tdi = ASPEED_JTAG_GET_TDI(xfer->direction, data[index]);
+ }
+ }
+
+ tdo = aspeed_jtag_tck_cycle(aspeed_jtag, 1, tdi & ASPEED_JTAG_DATA_MSB);
+ data[index] |= tdo << (shift_bits % ASPEED_JTAG_DATA_CHUNK_SIZE);
+
+ /* DIPause/DRPause */
+ aspeed_jtag_tck_cycle(aspeed_jtag, 0, 0);
+
+ if (xfer->endstate == JTAG_STATE_IDLE) {
+ /* ->DRExit2->DRUpdate->IDLE */
+ aspeed_jtag_sm_cycle(aspeed_jtag, sm_pause_idle,
+ sizeof(sm_pause_idle));
+ }
+}
+
+static void aspeed_jtag_xfer_push_data(struct aspeed_jtag *aspeed_jtag,
+ enum jtag_xfer_type type, u32 bits_len)
+{
+ if (type == JTAG_SIR_XFER) {
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_IOUT_LEN(bits_len),
+ ASPEED_JTAG_CTRL);
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_DOUT_LEN(bits_len) |
+ ASPEED_JTAG_CTL_INST_EN, ASPEED_JTAG_CTRL);
+ } else {
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_DOUT_LEN(bits_len),
+ ASPEED_JTAG_CTRL);
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_DOUT_LEN(bits_len) |
+ ASPEED_JTAG_CTL_DATA_EN, ASPEED_JTAG_CTRL);
+ }
+}
+
+static void aspeed_jtag_xfer_push_data_last(struct aspeed_jtag *aspeed_jtag,
+ enum jtag_xfer_type type,
+ u32 shift_bits,
+ enum jtag_endstate endstate)
+{
+ if (endstate == JTAG_STATE_IDLE) {
+ if (type == JTAG_SIR_XFER) {
+ aspeed_jtag_write(aspeed_jtag,
+ ASPEED_JTAG_IOUT_LEN(shift_bits),
+ ASPEED_JTAG_CTRL);
+ aspeed_jtag_write(aspeed_jtag,
+ ASPEED_JTAG_IOUT_LEN(shift_bits) |
+ ASPEED_JTAG_CTL_INST_EN,
+ ASPEED_JTAG_CTRL);
+ aspeed_jtag_wait_instruction_pause(aspeed_jtag);
+ } else {
+ aspeed_jtag_write(aspeed_jtag,
+ ASPEED_JTAG_DOUT_LEN(shift_bits) |
+ ASPEED_JTAG_CTL_DR_UPDATE,
+ ASPEED_JTAG_CTRL);
+ aspeed_jtag_write(aspeed_jtag,
+ ASPEED_JTAG_DOUT_LEN(shift_bits) |
+ ASPEED_JTAG_CTL_DR_UPDATE |
+ ASPEED_JTAG_CTL_DATA_EN,
+ ASPEED_JTAG_CTRL);
+ aspeed_jtag_wait_data_pause_complete(aspeed_jtag);
+ }
+ } else {
+ if (type == JTAG_SIR_XFER) {
+ aspeed_jtag_write(aspeed_jtag,
+ ASPEED_JTAG_IOUT_LEN(shift_bits) |
+ ASPEED_JTAG_CTL_LASPEED_INST,
+ ASPEED_JTAG_CTRL);
+ aspeed_jtag_write(aspeed_jtag,
+ ASPEED_JTAG_IOUT_LEN(shift_bits) |
+ ASPEED_JTAG_CTL_LASPEED_INST |
+ ASPEED_JTAG_CTL_INST_EN,
+ ASPEED_JTAG_CTRL);
+ aspeed_jtag_wait_instruction_complete(aspeed_jtag);
+ } else {
+ aspeed_jtag_write(aspeed_jtag,
+ ASPEED_JTAG_DOUT_LEN(shift_bits) |
+ ASPEED_JTAG_CTL_LASPEED_DATA,
+ ASPEED_JTAG_CTRL);
+ aspeed_jtag_write(aspeed_jtag,
+ ASPEED_JTAG_DOUT_LEN(shift_bits) |
+ ASPEED_JTAG_CTL_LASPEED_DATA |
+ ASPEED_JTAG_CTL_DATA_EN,
+ ASPEED_JTAG_CTRL);
+ aspeed_jtag_wait_data_complete(aspeed_jtag);
+ }
+ }
+}
+
+static void aspeed_jtag_xfer_hw(struct aspeed_jtag *aspeed_jtag,
+ struct jtag_xfer *xfer, u32 *data)
+{
+ unsigned long remain_xfer = xfer->length;
+ unsigned long index = 0;
+ char shift_bits;
+ u32 data_reg;
+
+ data_reg = xfer->type == JTAG_SIR_XFER ?
+ ASPEED_JTAG_INST : ASPEED_JTAG_DATA;
+ while (remain_xfer) {
+ if (xfer->direction == JTAG_WRITE_XFER)
+ aspeed_jtag_write(aspeed_jtag, data[index], data_reg);
+ else
+ aspeed_jtag_write(aspeed_jtag, 0, data_reg);
+
+ if (remain_xfer > ASPEED_JTAG_DATA_CHUNK_SIZE) {
+ shift_bits = ASPEED_JTAG_DATA_CHUNK_SIZE;
+
+ /*
+ * Read bytes were not equals to column length
+ * and go to Pause-DR
+ */
+ aspeed_jtag_xfer_push_data(aspeed_jtag, xfer->type,
+ shift_bits);
+ } else {
+ /*
+ * Read bytes equals to column length =>
+ * Update-DR
+ */
+ shift_bits = remain_xfer;
+ aspeed_jtag_xfer_push_data_last(aspeed_jtag, xfer->type,
+ shift_bits,
+ xfer->endstate);
+ }
+
+ if (xfer->direction == JTAG_READ_XFER) {
+ if (shift_bits < ASPEED_JTAG_DATA_CHUNK_SIZE) {
+ data[index] = aspeed_jtag_read(aspeed_jtag,
+ data_reg);
+
+ data[index] >>= ASPEED_JTAG_DATA_CHUNK_SIZE -
+ shift_bits;
+ } else {
+ data[index] = aspeed_jtag_read(aspeed_jtag,
+ data_reg);
+ }
+ }
+
+ remain_xfer = remain_xfer - shift_bits;
+ index++;
+ }
+}
+
+static int aspeed_jtag_xfer(struct jtag *jtag, struct jtag_xfer *xfer,
+ u8 *xfer_data)
+{
+ struct aspeed_jtag *aspeed_jtag = jtag_priv(jtag);
+
+ dev_dbg(aspeed_jtag->dev, "xfer %s\n",
+ xfer->type == JTAG_SIR_XFER ? "SIR" : "SDR");
+
+ if (!(aspeed_jtag->mode & JTAG_XFER_HW_MODE)) {
+ /* SW mode */
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_SW_TDIO, ASPEED_JTAG_SW);
+
+ aspeed_jtag_xfer_sw(aspeed_jtag, xfer, (u32 *)xfer_data);
+ } else {
+ /* HW mode */
+ aspeed_jtag_write(aspeed_jtag, 0, ASPEED_JTAG_SW);
+ aspeed_jtag_xfer_hw(aspeed_jtag, xfer, (u32 *)xfer_data);
+ }
+
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_SW_TDIO, ASPEED_JTAG_SW);
+ aspeed_jtag->status = xfer->endstate;
+ return 0;
+}
+
+static int aspeed_jtag_status_get(struct jtag *jtag, u32 *status)
+{
+ struct aspeed_jtag *aspeed_jtag = jtag_priv(jtag);
+
+ *status = aspeed_jtag->status;
+ return 0;
+}
+
+static irqreturn_t aspeed_jtag_interrupt(s32 this_irq, void *dev_id)
+{
+ struct aspeed_jtag *aspeed_jtag = dev_id;
+ irqreturn_t ret;
+ u32 status;
+
+ status = aspeed_jtag_read(aspeed_jtag, ASPEED_JTAG_ISR);
+
+ if (status & ASPEED_JTAG_ISR_INT_MASK) {
+ aspeed_jtag_write(aspeed_jtag,
+ (status & ASPEED_JTAG_ISR_INT_MASK)
+ | (status & ASPEED_JTAG_ISR_INT_EN_MASK),
+ ASPEED_JTAG_ISR);
+ aspeed_jtag->flag |= status & ASPEED_JTAG_ISR_INT_MASK;
+ }
+
+ if (aspeed_jtag->flag) {
+ wake_up_interruptible(&aspeed_jtag->jtag_wq);
+ ret = IRQ_HANDLED;
+ } else {
+ dev_err(aspeed_jtag->dev, "irq status:%x\n",
+ status);
+ ret = IRQ_NONE;
+ }
+ return ret;
+}
+
+static int aspeed_jtag_init(struct platform_device *pdev,
+ struct aspeed_jtag *aspeed_jtag)
+{
+ struct resource *res;
+ int err;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ aspeed_jtag->reg_base = devm_ioremap_resource(aspeed_jtag->dev, res);
+ if (IS_ERR(aspeed_jtag->reg_base))
+ return -ENOMEM;
+
+ aspeed_jtag->pclk = devm_clk_get(aspeed_jtag->dev, NULL);
+ if (IS_ERR(aspeed_jtag->pclk)) {
+ dev_err(aspeed_jtag->dev, "devm_clk_get failed\n");
+ return PTR_ERR(aspeed_jtag->pclk);
+ }
+
+ aspeed_jtag->irq = platform_get_irq(pdev, 0);
+ if (aspeed_jtag->irq < 0) {
+ dev_err(aspeed_jtag->dev, "no irq specified\n");
+ return -ENOENT;
+ }
+
+ if (clk_prepare_enable(aspeed_jtag->pclk)) {
+ dev_err(aspeed_jtag->dev, "no irq specified\n");
+ return -ENOENT;
+ }
+
+ aspeed_jtag->rst = devm_reset_control_get_shared(&pdev->dev, NULL);
+ if (IS_ERR(aspeed_jtag->rst)) {
+ dev_err(aspeed_jtag->dev,
+ "missing or invalid reset controller device tree entry");
+ return PTR_ERR(aspeed_jtag->rst);
+ }
+ reset_control_deassert(aspeed_jtag->rst);
+
+ /* Enable clock */
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_CTL_ENG_EN |
+ ASPEED_JTAG_CTL_ENG_OUT_EN, ASPEED_JTAG_CTRL);
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_SW_TDIO, ASPEED_JTAG_SW);
+
+ err = devm_request_irq(aspeed_jtag->dev, aspeed_jtag->irq,
+ aspeed_jtag_interrupt, 0,
+ "aspeed-jtag", aspeed_jtag);
+ if (err) {
+ dev_err(aspeed_jtag->dev, "unable to get IRQ");
+ goto clk_unprep;
+ }
+
+ aspeed_jtag_write(aspeed_jtag, ASPEED_JTAG_ISR_INST_PAUSE |
+ ASPEED_JTAG_ISR_INST_COMPLETE |
+ ASPEED_JTAG_ISR_DATA_PAUSE |
+ ASPEED_JTAG_ISR_DATA_COMPLETE |
+ ASPEED_JTAG_ISR_INST_PAUSE_EN |
+ ASPEED_JTAG_ISR_INST_COMPLETE_EN |
+ ASPEED_JTAG_ISR_DATA_PAUSE_EN |
+ ASPEED_JTAG_ISR_DATA_COMPLETE_EN,
+ ASPEED_JTAG_ISR);
+
+ aspeed_jtag->flag = 0;
+ aspeed_jtag->mode = 0;
+ init_waitqueue_head(&aspeed_jtag->jtag_wq);
+ return 0;
+
+clk_unprep:
+ clk_disable_unprepare(aspeed_jtag->pclk);
+ return err;
+}
+
+static int aspeed_jtag_deinit(struct platform_device *pdev,
+ struct aspeed_jtag *aspeed_jtag)
+{
+ aspeed_jtag_write(aspeed_jtag, 0, ASPEED_JTAG_ISR);
+ /* Disable clock */
+ aspeed_jtag_write(aspeed_jtag, 0, ASPEED_JTAG_CTRL);
+ reset_control_assert(aspeed_jtag->rst);
+ clk_disable_unprepare(aspeed_jtag->pclk);
+ return 0;
+}
+
+static const struct jtag_ops aspeed_jtag_ops = {
+ .freq_get = aspeed_jtag_freq_get,
+ .freq_set = aspeed_jtag_freq_set,
+ .status_get = aspeed_jtag_status_get,
+ .idle = aspeed_jtag_idle,
+ .xfer = aspeed_jtag_xfer,
+ .mode_set = aspeed_jtag_mode_set
+};
+
+static int aspeed_jtag_probe(struct platform_device *pdev)
+{
+ struct aspeed_jtag *aspeed_jtag;
+ struct jtag *jtag;
+ int err;
+
+ jtag = jtag_alloc(&pdev->dev, sizeof(*aspeed_jtag), &aspeed_jtag_ops);
+ if (!jtag)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, jtag);
+ aspeed_jtag = jtag_priv(jtag);
+ aspeed_jtag->dev = &pdev->dev;
+
+ /* Initialize device*/
+ err = aspeed_jtag_init(pdev, aspeed_jtag);
+ if (err)
+ goto err_jtag_init;
+
+ /* Initialize JTAG core structure*/
+ err = devm_jtag_register(aspeed_jtag->dev, jtag);
+ if (err)
+ goto err_jtag_register;
+
+ return 0;
+
+err_jtag_register:
+ aspeed_jtag_deinit(pdev, aspeed_jtag);
+err_jtag_init:
+ jtag_free(jtag);
+ return err;
+}
+
+static int aspeed_jtag_remove(struct platform_device *pdev)
+{
+ struct jtag *jtag = platform_get_drvdata(pdev);
+
+ aspeed_jtag_deinit(pdev, jtag_priv(jtag));
+ return 0;
+}
+
+static const struct of_device_id aspeed_jtag_of_match[] = {
+ { .compatible = "aspeed,ast2400-jtag", },
+ { .compatible = "aspeed,ast2500-jtag", },
+ {}
+};
+
+static struct platform_driver aspeed_jtag_driver = {
+ .probe = aspeed_jtag_probe,
+ .remove = aspeed_jtag_remove,
+ .driver = {
+ .name = ASPEED_JTAG_NAME,
+ .of_match_table = aspeed_jtag_of_match,
+ },
+};
+module_platform_driver(aspeed_jtag_driver);
+
+MODULE_AUTHOR("Oleksandr Shamray <oleksandrs@mellanox.com>");
+MODULE_DESCRIPTION("ASPEED JTAG driver");
+MODULE_LICENSE("GPL v2");
--
1.7.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox