* [PATCH V4 8/8] audit: initialize at subsystem time rather than device time
From: Richard Guy Briggs @ 2014-08-21 1:09 UTC (permalink / raw)
To: linux-audit, linux-kernel, containers, linux-api
Cc: Richard Guy Briggs, arozansk, eparis, sgrubb, ebiederm, serge
In-Reply-To: <cover.1408581429.git.rgb@redhat.com>
The audit subsystem should be initialized a bit earlier so that it is in place
in time for initial namespace serial number logging.
---
kernel/audit.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/kernel/audit.c b/kernel/audit.c
index 6d95d1c..aa99518 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -1186,7 +1186,7 @@ static int __init audit_init(void)
return 0;
}
-__initcall(audit_init);
+subsys_initcall(audit_init);
/* Process kernel command-line parameter at boot time. audit=0 or audit=1. */
static int __init audit_enable(char *str)
--
1.7.1
^ permalink raw reply related
* [PATCH V4 7/8] audit: log creation and deletion of namespace instances
From: Richard Guy Briggs @ 2014-08-21 1:09 UTC (permalink / raw)
To: linux-audit-H+wXaHxf7aLQT0dZR+AlfA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-api-u79uwXL29TY76Z2rM5mHXA
Cc: ebiederm-aS9lmoZGLiVWk0Htik3J/w, eparis-H+wXaHxf7aLQT0dZR+AlfA,
sgrubb-H+wXaHxf7aLQT0dZR+AlfA
In-Reply-To: <cover.1408581429.git.rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Log the creation and deletion of namespace instances in all 6 types of
namespaces.
Twelve new audit message types have been introduced:
AUDIT_NS_INIT_MNT 1330 /* Record mount namespace instance creation */
AUDIT_NS_INIT_UTS 1331 /* Record UTS namespace instance creation */
AUDIT_NS_INIT_IPC 1332 /* Record IPC namespace instance creation */
AUDIT_NS_INIT_USER 1333 /* Record USER namespace instance creation */
AUDIT_NS_INIT_PID 1334 /* Record PID namespace instance creation */
AUDIT_NS_INIT_NET 1335 /* Record NET namespace instance creation */
AUDIT_NS_DEL_MNT 1336 /* Record mount namespace instance deletion */
AUDIT_NS_DEL_UTS 1337 /* Record UTS namespace instance deletion */
AUDIT_NS_DEL_IPC 1338 /* Record IPC namespace instance deletion */
AUDIT_NS_DEL_USER 1339 /* Record USER namespace instance deletion */
AUDIT_NS_DEL_PID 1340 /* Record PID namespace instance deletion */
AUDIT_NS_DEL_NET 1341 /* Record NET namespace instance deletion */
As suggested by Eric Paris, there are 12 message types, one for each of
creation and deletion, one for each type of namespace so that text searches are
easier in conjunction with the AUDIT_NS_INFO message type, being able to search
for all records such as "netns=7 " and to avoid fields disappearing per message
type to make ausearch more efficient.
A typical startup would look roughly like:
type=AUDIT_NS_INIT_UTS msg=audit(1408577534.868:5): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel old_utsns=0 utsns=2 res=1
type=AUDIT_NS_INIT_USER msg=audit(1408577534.868:6): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel old_userns=0 userns=3 res=1
type=AUDIT_NS_INIT_PID msg=audit(1408577534.868:7): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel old_pidns=0 pidns=4 res=1
type=AUDIT_NS_INIT_MNT msg=audit(1408577534.868:8): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel old_mntns=0 mntns=5 res=1
type=AUDIT_NS_INIT_IPC msg=audit(1408577534.868:9): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel old_ipcns=0 ipcns=1 res=1
type=AUDIT_NS_INIT_NET msg=audit(1408577533.500:10): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel old_netns=0 netns=7 res=1
And a CLONE action would result in:
type=type=AUDIT_NS_INIT_NET msg=audit(1408577535.306:81): pid=481 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 old_netns=7 netns=8 res=1
type=type=AUDIT_NS_INIT_MNT msg=audit(1408577535.307:83): pid=481 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 old_mntns=5 mntns=9 res=1
While deleting a namespace would result in:
type=type=AUDIT_NS_DEL_MNT msg=audit(1408577552.221:85): pid=481 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 mntns=9 res=1
If non-zero, old_snum lists the namespace from which it was cloned.
Signed-off-by: Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/namespace.c | 12 +++++++++++
include/linux/audit.h | 8 +++++++
include/uapi/linux/audit.h | 12 +++++++++++
ipc/namespace.c | 10 +++++++++
kernel/audit.c | 47 ++++++++++++++++++++++++++++++++++++++++++++
kernel/pid_namespace.c | 10 +++++++++
kernel/user_namespace.c | 11 ++++++++++
kernel/utsname.c | 11 ++++++++++
net/core/net_namespace.c | 12 +++++++++++
9 files changed, 133 insertions(+), 0 deletions(-)
diff --git a/fs/namespace.c b/fs/namespace.c
index f433f21..cb05b3d 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -24,6 +24,7 @@
#include <linux/proc_ns.h>
#include <linux/magic.h>
#include <linux/bootmem.h>
+#include <linux/audit.h>
#include "pnode.h"
#include "internal.h"
@@ -2459,6 +2460,7 @@ dput_out:
static void free_mnt_ns(struct mnt_namespace *ns)
{
+ audit_log_ns_del(AUDIT_NS_DEL_MNT, ns->serial_num);
proc_free_inum(ns->proc_inum);
put_user_ns(ns->user_ns);
kfree(ns);
@@ -2519,6 +2521,7 @@ struct mnt_namespace *copy_mnt_ns(unsigned long flags, struct mnt_namespace *ns,
new_ns = alloc_mnt_ns(user_ns);
if (IS_ERR(new_ns))
return new_ns;
+ audit_log_ns_init(AUDIT_NS_INIT_MNT, ns->serial_num, new_ns->serial_num);
namespace_lock();
/* First pass: copy the tree topology */
@@ -2831,6 +2834,15 @@ static void __init init_mount_tree(void)
set_fs_root(current->fs, &root);
}
+/* log the serial number of init mnt namespace after audit service starts */
+static int __init mnt_ns_init_log(void)
+{
+ struct mnt_namespace *init_mnt_ns = init_task.nsproxy->mnt_ns;
+ audit_log_ns_init(AUDIT_NS_INIT_MNT, 0, init_mnt_ns->serial_num);
+ return 0;
+}
+late_initcall(mnt_ns_init_log);
+
void __init mnt_init(void)
{
unsigned u;
diff --git a/include/linux/audit.h b/include/linux/audit.h
index 5ea3609..c245837 100644
--- a/include/linux/audit.h
+++ b/include/linux/audit.h
@@ -466,6 +466,9 @@ extern void audit_log_key(struct audit_buffer *ab,
char *key);
extern void audit_log_link_denied(const char *operation,
struct path *link);
+extern int audit_log_ns_init(int type, long long old_snum,
+ long long snum);
+extern int audit_log_ns_del(int type, long long snum);
extern void audit_log_lost(const char *message);
#ifdef CONFIG_SECURITY
extern void audit_log_secctx(struct audit_buffer *ab, u32 secid);
@@ -524,6 +527,11 @@ static inline void audit_log_key(struct audit_buffer *ab, char *key)
static inline void audit_log_link_denied(const char *string,
const struct path *link)
{ }
+static inline int audit_log_ns_init(int type, long long old_snum,
+ long long snum)
+{ }
+static inline int audit_log_ns_del(int type, long long snum)
+{ }
static inline void audit_log_secctx(struct audit_buffer *ab, u32 secid)
{ }
static inline int audit_log_task_context(struct audit_buffer *ab)
diff --git a/include/uapi/linux/audit.h b/include/uapi/linux/audit.h
index 84bbcdb..fd13a02 100644
--- a/include/uapi/linux/audit.h
+++ b/include/uapi/linux/audit.h
@@ -111,6 +111,18 @@
#define AUDIT_PROCTITLE 1327 /* Proctitle emit event */
#define AUDIT_FEATURE_CHANGE 1328 /* audit log listing feature changes */
#define AUDIT_NS_INFO 1329 /* Record process namespace IDs */
+#define AUDIT_NS_INIT_MNT 1330 /* Record mount namespace instance creation */
+#define AUDIT_NS_INIT_UTS 1331 /* Record UTS namespace instance creation */
+#define AUDIT_NS_INIT_IPC 1332 /* Record IPC namespace instance creation */
+#define AUDIT_NS_INIT_USER 1333 /* Record USER namespace instance creation */
+#define AUDIT_NS_INIT_PID 1334 /* Record PID namespace instance creation */
+#define AUDIT_NS_INIT_NET 1335 /* Record NET namespace instance creation */
+#define AUDIT_NS_DEL_MNT 1336 /* Record mount namespace instance deletion */
+#define AUDIT_NS_DEL_UTS 1337 /* Record UTS namespace instance deletion */
+#define AUDIT_NS_DEL_IPC 1338 /* Record IPC namespace instance deletion */
+#define AUDIT_NS_DEL_USER 1339 /* Record USER namespace instance deletion */
+#define AUDIT_NS_DEL_PID 1340 /* Record PID namespace instance deletion */
+#define AUDIT_NS_DEL_NET 1341 /* Record NET namespace instance deletion */
#define AUDIT_AVC 1400 /* SE Linux avc denial or grant */
#define AUDIT_SELINUX_ERR 1401 /* Internal SE Linux Errors */
diff --git a/ipc/namespace.c b/ipc/namespace.c
index 36ce7ff..538c4b9 100644
--- a/ipc/namespace.c
+++ b/ipc/namespace.c
@@ -13,6 +13,7 @@
#include <linux/mount.h>
#include <linux/user_namespace.h>
#include <linux/proc_ns.h>
+#include <linux/audit.h>
#include "util.h"
@@ -42,6 +43,7 @@ static struct ipc_namespace *create_ipc_ns(struct user_namespace *user_ns,
atomic_inc(&nr_ipc_ns);
ns->serial_num = ns_serial();
+ audit_log_ns_init(AUDIT_NS_INIT_IPC, old_ns->serial_num, ns->serial_num);
sem_init_ns(ns);
msg_init_ns(ns);
@@ -121,6 +123,7 @@ static void free_ipc_ns(struct ipc_namespace *ns)
*/
ipcns_notify(IPCNS_REMOVED);
put_user_ns(ns->user_ns);
+ audit_log_ns_del(AUDIT_NS_DEL_IPC, ns->serial_num);
proc_free_inum(ns->proc_inum);
kfree(ns);
}
@@ -207,3 +210,10 @@ const struct proc_ns_operations ipcns_operations = {
.inum = ipcns_inum,
.snum = ipcns_snum,
};
+
+static int __init ipc_namespaces_init(void)
+{
+ return audit_log_ns_init(AUDIT_NS_INIT_IPC, 0, init_ipc_ns.serial_num);
+}
+
+late_initcall(ipc_namespaces_init);
diff --git a/kernel/audit.c b/kernel/audit.c
index a4c39a0..6d95d1c 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -1970,6 +1970,53 @@ out:
kfree(name);
}
+static char* ns_name[] = {
+ "mnt",
+ "uts",
+ "ipc",
+ "user",
+ "pid",
+ "net",
+};
+
+/**
+ * audit_log_ns_init - report a namespace instance creation
+ * @type: type of audit namespace instance created message
+ * @old_snum: the serial number of the cloned namespace instance
+ * @snum: the serial number of the new namespace instance
+ */
+int audit_log_ns_init(int type, long long old_snum, long long snum)
+{
+ struct audit_buffer *ab;
+ char* audit_ns_name = ns_name[type - AUDIT_NS_INIT_MNT];
+
+ if (type < AUDIT_NS_INIT_MNT || type > AUDIT_NS_INIT_NET)
+ return 0;
+ audit_log_common_recv_msg(&ab, type);
+ audit_log_format(ab, " old_%sns=%llx %sns=%llx res=1",
+ audit_ns_name, old_snum, audit_ns_name, snum);
+ audit_log_end(ab);
+ return 0;
+}
+
+/**
+ * audit_log_ns_del - report a namespace instance deleted
+ * @type: type of audit namespace instance deleted message
+ * @snum: the serial number of the namespace instance
+ */
+int audit_log_ns_del(int type, long long snum)
+{
+ struct audit_buffer *ab;
+ char* audit_ns_name = ns_name[type - AUDIT_NS_DEL_MNT];
+
+ if (type < AUDIT_NS_DEL_MNT || type > AUDIT_NS_DEL_NET)
+ return 0;
+ audit_log_common_recv_msg(&ab, type);
+ audit_log_format(ab, " %ssn=%llx res=1", audit_ns_name, snum);
+ audit_log_end(ab);
+ return 0;
+}
+
/**
* audit_log_end - end one audit record
* @ab: the audit_buffer
diff --git a/kernel/pid_namespace.c b/kernel/pid_namespace.c
index 059b330..7da351a 100644
--- a/kernel/pid_namespace.c
+++ b/kernel/pid_namespace.c
@@ -18,6 +18,7 @@
#include <linux/proc_ns.h>
#include <linux/reboot.h>
#include <linux/export.h>
+#include <linux/audit.h>
struct pid_cache {
int nr_ids;
@@ -110,6 +111,8 @@ static struct pid_namespace *create_pid_namespace(struct user_namespace *user_ns
goto out_free_map;
ns->serial_num = ns_serial();
+ audit_log_ns_init(AUDIT_NS_INIT_PID, parent_pid_ns->serial_num,
+ ns->serial_num);
kref_init(&ns->kref);
ns->level = level;
@@ -144,6 +147,7 @@ static void destroy_pid_namespace(struct pid_namespace *ns)
{
int i;
+ audit_log_ns_del(AUDIT_NS_DEL_PID, ns->serial_num);
proc_free_inum(ns->proc_inum);
for (i = 0; i < PIDMAP_ENTRIES; i++)
kfree(ns->pidmap[i].page);
@@ -397,3 +401,9 @@ static __init int pid_namespaces_init(void)
}
__initcall(pid_namespaces_init);
+
+static __init int pid_namespaces_late_init(void)
+{
+ return audit_log_ns_init(AUDIT_NS_INIT_PID, 0, init_pid_ns.serial_num);
+}
+late_initcall(pid_namespaces_late_init);
diff --git a/kernel/user_namespace.c b/kernel/user_namespace.c
index 3f04df5..24497a0 100644
--- a/kernel/user_namespace.c
+++ b/kernel/user_namespace.c
@@ -22,6 +22,7 @@
#include <linux/ctype.h>
#include <linux/projid.h>
#include <linux/fs_struct.h>
+#include <linux/audit.h>
static struct kmem_cache *user_ns_cachep __read_mostly;
@@ -93,6 +94,8 @@ int create_user_ns(struct cred *new)
}
ns->serial_num = ns_serial();
+ audit_log_ns_init(AUDIT_NS_INIT_USER, parent_ns->serial_num,
+ ns->serial_num);
atomic_set(&ns->count, 1);
/* Leave the new->user_ns reference with the new user namespace. */
@@ -138,6 +141,7 @@ void free_user_ns(struct user_namespace *ns)
#ifdef CONFIG_PERSISTENT_KEYRINGS
key_put(ns->persistent_keyring_register);
#endif
+ audit_log_ns_del(AUDIT_NS_DEL_USER, ns->serial_num);
proc_free_inum(ns->proc_inum);
kmem_cache_free(user_ns_cachep, ns);
ns = parent;
@@ -918,3 +922,10 @@ static __init int user_namespaces_init(void)
return 0;
}
subsys_initcall(user_namespaces_init);
+
+static __init int user_namespaces_late_init(void)
+{
+ return audit_log_ns_init(AUDIT_NS_INIT_USER, 0,
+ init_user_ns.serial_num);
+}
+late_initcall(user_namespaces_late_init);
diff --git a/kernel/utsname.c b/kernel/utsname.c
index ffeac1b..05ecc2d 100644
--- a/kernel/utsname.c
+++ b/kernel/utsname.c
@@ -16,6 +16,7 @@
#include <linux/slab.h>
#include <linux/user_namespace.h>
#include <linux/proc_ns.h>
+#include <linux/audit.h>
static struct uts_namespace *create_uts_ns(void)
{
@@ -49,6 +50,7 @@ static struct uts_namespace *clone_uts_ns(struct user_namespace *user_ns,
}
ns->serial_num = ns_serial();
+ audit_log_ns_init(AUDIT_NS_INIT_UTS, old_ns->serial_num, ns->serial_num);
down_read(&uts_sem);
memcpy(&ns->name, &old_ns->name, sizeof(ns->name));
@@ -86,6 +88,7 @@ void free_uts_ns(struct kref *kref)
ns = container_of(kref, struct uts_namespace, kref);
put_user_ns(ns->user_ns);
+ audit_log_ns_del(AUDIT_NS_DEL_UTS, ns->serial_num);
proc_free_inum(ns->proc_inum);
kfree(ns);
}
@@ -148,3 +151,11 @@ const struct proc_ns_operations utsns_operations = {
.inum = utsns_inum,
.snum = utsns_snum,
};
+
+static int __init uts_namespaces_init(void)
+{
+ return audit_log_ns_init(AUDIT_NS_INIT_UTS, 0,
+ init_uts_ns.serial_num);
+}
+
+late_initcall(uts_namespaces_init);
diff --git a/net/core/net_namespace.c b/net/core/net_namespace.c
index c402eea..295ecc3 100644
--- a/net/core/net_namespace.c
+++ b/net/core/net_namespace.c
@@ -17,6 +17,7 @@
#include <linux/user_namespace.h>
#include <net/net_namespace.h>
#include <net/netns/generic.h>
+#include <linux/audit.h>
/*
* Our network namespace constructor/destructor lists
@@ -253,6 +254,8 @@ struct net *copy_net_ns(unsigned long flags,
mutex_lock(&net_mutex);
rv = setup_net(net, user_ns);
if (rv == 0) {
+ audit_log_ns_init(AUDIT_NS_INIT_NET, old_net->serial_num,
+ net->serial_num);
rtnl_lock();
list_add_tail_rcu(&net->list, &net_namespace_list);
rtnl_unlock();
@@ -395,6 +398,7 @@ static __net_init int net_ns_net_init(struct net *net)
static __net_exit void net_ns_net_exit(struct net *net)
{
+ audit_log_ns_del(AUDIT_NS_DEL_NET, net->serial_num);
proc_free_inum(net->proc_inum);
}
@@ -441,6 +445,14 @@ static int __init net_ns_init(void)
pure_initcall(net_ns_init);
+/* log the serial number of init_net namespace after audit service starts */
+static int __init net_ns_init_log(void)
+{
+ audit_log_ns_init(AUDIT_NS_INIT_NET, 0, init_net.serial_num);
+ return 0;
+}
+late_initcall(net_ns_init_log);
+
#ifdef CONFIG_NET_NS
static int __register_pernet_operations(struct list_head *list,
struct pernet_operations *ops)
--
1.7.1
^ permalink raw reply related
* [PATCH V4 6/8] audit: log namespace serial numbers
From: Richard Guy Briggs @ 2014-08-21 1:09 UTC (permalink / raw)
To: linux-audit-H+wXaHxf7aLQT0dZR+AlfA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-api-u79uwXL29TY76Z2rM5mHXA
Cc: ebiederm-aS9lmoZGLiVWk0Htik3J/w, eparis-H+wXaHxf7aLQT0dZR+AlfA,
sgrubb-H+wXaHxf7aLQT0dZR+AlfA
In-Reply-To: <cover.1408581429.git.rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Log the namespace serial numbers of a task in a new record type (1329) (usually
accompanies audit_log_task_info() type=SYSCALL record) which is used by syscall
audits, among others..
Idea first presented:
https://www.redhat.com/archives/linux-audit/2013-March/msg00020.html
Typical output format would look something like:
type=NS_INFO msg=audit(1408577535.306:82): netns=8 utsns=2 ipcns=1 pidns=4 userns=3 mntns=5
The serial numbers are printed in hex.
Suggested-by: Aristeu Rozanski <arozansk-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Acked-by: Serge Hallyn <serge.hallyn-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>
---
include/linux/audit.h | 7 +++++++
include/uapi/linux/audit.h | 1 +
kernel/audit.c | 29 +++++++++++++++++++++++++++++
kernel/auditsc.c | 2 ++
security/integrity/ima/ima_api.c | 2 ++
5 files changed, 41 insertions(+), 0 deletions(-)
diff --git a/include/linux/audit.h b/include/linux/audit.h
index 22cfddb..5ea3609 100644
--- a/include/linux/audit.h
+++ b/include/linux/audit.h
@@ -101,6 +101,13 @@ extern int __weak audit_classify_compat_syscall(int abi, unsigned syscall);
struct filename;
extern void audit_log_session_info(struct audit_buffer *ab);
+#ifdef CONFIG_NAMESPACES
+extern void audit_log_namespace_info(struct task_struct *tsk);
+#else
+void audit_log_namespace_info(struct task_struct *tsk)
+{
+}
+#endif
#ifdef CONFIG_AUDIT_COMPAT_GENERIC
#define audit_is_compat(arch) (!((arch) & __AUDIT_ARCH_64BIT))
diff --git a/include/uapi/linux/audit.h b/include/uapi/linux/audit.h
index cf67147..84bbcdb 100644
--- a/include/uapi/linux/audit.h
+++ b/include/uapi/linux/audit.h
@@ -110,6 +110,7 @@
#define AUDIT_SECCOMP 1326 /* Secure Computing event */
#define AUDIT_PROCTITLE 1327 /* Proctitle emit event */
#define AUDIT_FEATURE_CHANGE 1328 /* audit log listing feature changes */
+#define AUDIT_NS_INFO 1329 /* Record process namespace IDs */
#define AUDIT_AVC 1400 /* SE Linux avc denial or grant */
#define AUDIT_SELINUX_ERR 1401 /* Internal SE Linux Errors */
diff --git a/kernel/audit.c b/kernel/audit.c
index 3ef2e0e..a4c39a0 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -65,6 +65,7 @@
#include <linux/freezer.h>
#include <linux/tty.h>
#include <linux/pid_namespace.h>
+#include <linux/proc_ns.h>
#include <net/netns/generic.h>
#include "audit.h"
@@ -743,6 +744,8 @@ static void audit_log_feature_change(int which, u32 old_feature, u32 new_feature
audit_feature_names[which], !!old_feature, !!new_feature,
!!old_lock, !!new_lock, res);
audit_log_end(ab);
+
+ audit_log_namespace_info(current);
}
static int audit_set_feature(struct sk_buff *skb)
@@ -1661,6 +1664,30 @@ void audit_log_session_info(struct audit_buffer *ab)
audit_log_format(ab, " auid=%u ses=%u", auid, sessionid);
}
+#ifdef CONFIG_NAMESPACES
+void audit_log_namespace_info(struct task_struct *tsk)
+{
+ const struct proc_ns_operations **entry;
+ bool end = false;
+ struct audit_buffer *ab;
+
+ if (!tsk)
+ return;
+ ab = audit_log_start(tsk->audit_context, GFP_KERNEL,
+ AUDIT_NS_INFO);
+ if (!ab)
+ return;
+ for (entry = ns_entries; !end; entry++) {
+ void *ns = (*entry)->get(tsk);
+ audit_log_format(ab, " %sns=%llx", (*entry)->name,
+ (*entry)->snum(ns));
+ (*entry)->put(ns);
+ end = (*entry)->type == CLONE_NEWNS;
+ }
+ audit_log_end(ab);
+}
+#endif /* CONFIG_NAMESPACES */
+
void audit_log_key(struct audit_buffer *ab, char *key)
{
audit_log_format(ab, " key=");
@@ -1933,6 +1960,8 @@ void audit_log_link_denied(const char *operation, struct path *link)
audit_log_format(ab, " res=0");
audit_log_end(ab);
+ audit_log_namespace_info(current);
+
/* Generate AUDIT_PATH record with object. */
name->type = AUDIT_TYPE_NORMAL;
audit_copy_inode(name, link->dentry, link->dentry->d_inode);
diff --git a/kernel/auditsc.c b/kernel/auditsc.c
index 21eae3c..08b9af9 100644
--- a/kernel/auditsc.c
+++ b/kernel/auditsc.c
@@ -1383,6 +1383,8 @@ static void audit_log_exit(struct audit_context *context, struct task_struct *ts
audit_log_key(ab, context->filterkey);
audit_log_end(ab);
+ audit_log_namespace_info(tsk);
+
for (aux = context->aux; aux; aux = aux->next) {
ab = audit_log_start(context, GFP_KERNEL, aux->type);
diff --git a/security/integrity/ima/ima_api.c b/security/integrity/ima/ima_api.c
index d9cd5ce..06d6897 100644
--- a/security/integrity/ima/ima_api.c
+++ b/security/integrity/ima/ima_api.c
@@ -323,6 +323,8 @@ void ima_audit_measurement(struct integrity_iint_cache *iint,
audit_log_task_info(ab, current);
audit_log_end(ab);
+ audit_log_namespace_info(current);
+
iint->flags |= IMA_AUDITED;
}
--
1.7.1
^ permalink raw reply related
* [PATCH V4 5/8] namespaces: expose ns_entries
From: Richard Guy Briggs @ 2014-08-21 1:09 UTC (permalink / raw)
To: linux-audit-H+wXaHxf7aLQT0dZR+AlfA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-api-u79uwXL29TY76Z2rM5mHXA
Cc: ebiederm-aS9lmoZGLiVWk0Htik3J/w, eparis-H+wXaHxf7aLQT0dZR+AlfA,
sgrubb-H+wXaHxf7aLQT0dZR+AlfA
In-Reply-To: <cover.1408581429.git.rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Expose ns_entries so subsystems other than proc can use this set of namespace
operations.
Signed-off-by: Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/proc/namespaces.c | 2 +-
include/linux/proc_ns.h | 1 +
2 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/fs/proc/namespaces.c b/fs/proc/namespaces.c
index e953e0a..29c3909 100644
--- a/fs/proc/namespaces.c
+++ b/fs/proc/namespaces.c
@@ -15,7 +15,7 @@
#include "internal.h"
-static const struct proc_ns_operations *ns_entries[] = {
+const struct proc_ns_operations *ns_entries[] = {
#ifdef CONFIG_NET_NS
&netns_operations,
#endif
diff --git a/include/linux/proc_ns.h b/include/linux/proc_ns.h
index aaafe3e..f4563db 100644
--- a/include/linux/proc_ns.h
+++ b/include/linux/proc_ns.h
@@ -28,6 +28,7 @@ extern const struct proc_ns_operations ipcns_operations;
extern const struct proc_ns_operations pidns_operations;
extern const struct proc_ns_operations userns_operations;
extern const struct proc_ns_operations mntns_operations;
+extern const struct proc_ns_operations *ns_entries[];
/*
* We always define these enumerators
--
1.7.1
^ permalink raw reply related
* [PATCH V4 4/8] Documentation: add a section for /proc/<pid>/ns/
From: Richard Guy Briggs @ 2014-08-21 1:09 UTC (permalink / raw)
To: linux-audit-H+wXaHxf7aLQT0dZR+AlfA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-api-u79uwXL29TY76Z2rM5mHXA
Cc: ebiederm-aS9lmoZGLiVWk0Htik3J/w, eparis-H+wXaHxf7aLQT0dZR+AlfA,
sgrubb-H+wXaHxf7aLQT0dZR+AlfA
In-Reply-To: <cover.1408581429.git.rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
Documentation/filesystems/proc.txt | 16 ++++++++++++++++
1 files changed, 16 insertions(+), 0 deletions(-)
diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt
index ddc531a..c4bfd6f 100644
--- a/Documentation/filesystems/proc.txt
+++ b/Documentation/filesystems/proc.txt
@@ -42,6 +42,7 @@ Table of Contents
3.6 /proc/<pid>/comm & /proc/<pid>/task/<tid>/comm
3.7 /proc/<pid>/task/<tid>/children - Information about task children
3.8 /proc/<pid>/fdinfo/<fd> - Information about opened file
+ 3.9 /proc/<pid>/ns/<ns>{,_snum} - Information about process namespaces
4 Configuring procfs
4.1 Mount options
@@ -1744,6 +1745,21 @@ pair provide additional information particular to the objects they represent.
optional and may be omitted if no marks created yet.
+3.9 /proc/<pid>/ns/<nstype>{,_snum} - Information about process namespaces
+--------------------------------------------------------------------------
+These files provides information about the namespaces within which the process
+is contained. The files named only with the namespace type <nstype> contain a
+link that lists the containing namespace' inode number in its proc filesystem.
+The files with suffix _snum contain a link that lists the containing
+namespace' instance serial number, unique per kernel since boot. The
+namespace types are self-describing.
+
+The output format of the inode links is:
+ <nstype>:[<inode_number>]
+The output format of the serial number links is:
+ <nstype>_snum:[<serial_number>]
+
+
------------------------------------------------------------------------------
Configuring procfs
------------------------------------------------------------------------------
--
1.7.1
^ permalink raw reply related
* [PATCH V4 3/8] namespaces: expose ns instance serial numbers in proc
From: Richard Guy Briggs @ 2014-08-21 1:09 UTC (permalink / raw)
To: linux-audit-H+wXaHxf7aLQT0dZR+AlfA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-api-u79uwXL29TY76Z2rM5mHXA
Cc: ebiederm-aS9lmoZGLiVWk0Htik3J/w, eparis-H+wXaHxf7aLQT0dZR+AlfA,
sgrubb-H+wXaHxf7aLQT0dZR+AlfA
In-Reply-To: <cover.1408581429.git.rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Expose the namespace instace serial numbers in the proc filesystem at
/proc/<pid>/ns/<ns>_snum. The link text gives the serial number in hex.
"snum" was chosen instead of "seq" for consistency with inum and there are a
number of other uses of "seq" in the namespace code.
Suggested-by: Serge E. Hallyn <serge-A9i7LUbDfNHQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/proc/namespaces.c | 33 +++++++++++++++++++++++++--------
1 files changed, 25 insertions(+), 8 deletions(-)
diff --git a/fs/proc/namespaces.c b/fs/proc/namespaces.c
index 8902609..e953e0a 100644
--- a/fs/proc/namespaces.c
+++ b/fs/proc/namespaces.c
@@ -47,12 +47,15 @@ static char *ns_dname(struct dentry *dentry, char *buffer, int buflen)
struct inode *inode = dentry->d_inode;
const struct proc_ns_operations *ns_ops = PROC_I(inode)->ns.ns_ops;
- return dynamic_dname(dentry, buffer, buflen, "%s:[%lu]",
- ns_ops->name, inode->i_ino);
+ if (strstr(dentry->d_iname, "_snum"))
+ return dynamic_dname(dentry, buffer, buflen, "%s_snum:[%llx]",
+ ns_ops->name, ns_ops->snum(PROC_I(inode)->ns.ns));
+ else
+ return dynamic_dname(dentry, buffer, buflen, "%s:[%lu]",
+ ns_ops->name, inode->i_ino);
}
-const struct dentry_operations ns_dentry_operations =
-{
+const struct dentry_operations ns_dentry_operations = {
.d_delete = always_delete_dentry,
.d_dname = ns_dname,
};
@@ -160,7 +163,10 @@ static int proc_ns_readlink(struct dentry *dentry, char __user *buffer, int bufl
if (!ns)
goto out_put_task;
- snprintf(name, sizeof(name), "%s:[%u]", ns_ops->name, ns_ops->inum(ns));
+ if (strstr(dentry->d_iname, "_snum"))
+ snprintf(name, sizeof(name), "%s_snum:[%llx]", ns_ops->name, ns_ops->snum(ns));
+ else
+ snprintf(name, sizeof(name), "%s:[%u]", ns_ops->name, ns_ops->inum(ns));
res = readlink_copy(buffer, buflen, name);
ns_ops->put(ns);
out_put_task:
@@ -210,16 +216,23 @@ static int proc_ns_dir_readdir(struct file *file, struct dir_context *ctx)
if (!dir_emit_dots(file, ctx))
goto out;
- if (ctx->pos >= 2 + ARRAY_SIZE(ns_entries))
+ if (ctx->pos >= 2 + 2 * ARRAY_SIZE(ns_entries))
goto out;
entry = ns_entries + (ctx->pos - 2);
last = &ns_entries[ARRAY_SIZE(ns_entries) - 1];
while (entry <= last) {
const struct proc_ns_operations *ops = *entry;
+ char name[50];
+
if (!proc_fill_cache(file, ctx, ops->name, strlen(ops->name),
proc_ns_instantiate, task, ops))
break;
ctx->pos++;
+ snprintf(name, sizeof(name), "%s_snum", ops->name);
+ if (!proc_fill_cache(file, ctx, name, strlen(name),
+ proc_ns_instantiate, task, ops))
+ break;
+ ctx->pos++;
entry++;
}
out:
@@ -247,9 +260,13 @@ static struct dentry *proc_ns_dir_lookup(struct inode *dir,
last = &ns_entries[ARRAY_SIZE(ns_entries)];
for (entry = ns_entries; entry < last; entry++) {
- if (strlen((*entry)->name) != len)
+ char name[50];
+
+ snprintf(name, sizeof(name), "%s_snum", (*entry)->name);
+ if (strlen((*entry)->name) != len && strlen(name) != len)
continue;
- if (!memcmp(dentry->d_name.name, (*entry)->name, len))
+ if (!memcmp(dentry->d_name.name, (*entry)->name, len)
+ || !memcmp(dentry->d_name.name, name, len))
break;
}
if (entry == last)
--
1.7.1
^ permalink raw reply related
* [PATCH V4 2/8] namespaces: expose namespace instance serial number in proc_ns_operations
From: Richard Guy Briggs @ 2014-08-21 1:09 UTC (permalink / raw)
To: linux-audit-H+wXaHxf7aLQT0dZR+AlfA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-api-u79uwXL29TY76Z2rM5mHXA
Cc: ebiederm-aS9lmoZGLiVWk0Htik3J/w, eparis-H+wXaHxf7aLQT0dZR+AlfA,
sgrubb-H+wXaHxf7aLQT0dZR+AlfA
In-Reply-To: <cover.1408581429.git.rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Expose the namespace instance serial number for each namespace type in the proc
namespace operations structure to make it available for the proc filesystem.
Signed-off-by: Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/namespace.c | 7 +++++++
include/linux/proc_ns.h | 1 +
ipc/namespace.c | 8 ++++++++
kernel/pid_namespace.c | 7 +++++++
kernel/user_namespace.c | 7 +++++++
kernel/utsname.c | 8 ++++++++
net/core/net_namespace.c | 7 +++++++
7 files changed, 45 insertions(+), 0 deletions(-)
diff --git a/fs/namespace.c b/fs/namespace.c
index 9af49ff..f433f21 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -3028,6 +3028,12 @@ static unsigned int mntns_inum(void *ns)
return mnt_ns->proc_inum;
}
+static long long mntns_snum(void *ns)
+{
+ struct mnt_namespace *mnt_ns = ns;
+ return mnt_ns->serial_num;
+}
+
const struct proc_ns_operations mntns_operations = {
.name = "mnt",
.type = CLONE_NEWNS,
@@ -3035,4 +3041,5 @@ const struct proc_ns_operations mntns_operations = {
.put = mntns_put,
.install = mntns_install,
.inum = mntns_inum,
+ .snum = mntns_snum,
};
diff --git a/include/linux/proc_ns.h b/include/linux/proc_ns.h
index 34a1e10..aaafe3e 100644
--- a/include/linux/proc_ns.h
+++ b/include/linux/proc_ns.h
@@ -14,6 +14,7 @@ struct proc_ns_operations {
void (*put)(void *ns);
int (*install)(struct nsproxy *nsproxy, void *ns);
unsigned int (*inum)(void *ns);
+ long long (*snum)(void *ns);
};
struct proc_ns {
diff --git a/ipc/namespace.c b/ipc/namespace.c
index 76dac5c..36ce7ff 100644
--- a/ipc/namespace.c
+++ b/ipc/namespace.c
@@ -191,6 +191,13 @@ static unsigned int ipcns_inum(void *vp)
return ns->proc_inum;
}
+static long long ipcns_snum(void *vp)
+{
+ struct ipc_namespace *ns = vp;
+
+ return ns->serial_num;
+}
+
const struct proc_ns_operations ipcns_operations = {
.name = "ipc",
.type = CLONE_NEWIPC,
@@ -198,4 +205,5 @@ const struct proc_ns_operations ipcns_operations = {
.put = ipcns_put,
.install = ipcns_install,
.inum = ipcns_inum,
+ .snum = ipcns_snum,
};
diff --git a/kernel/pid_namespace.c b/kernel/pid_namespace.c
index 40a8b36..059b330 100644
--- a/kernel/pid_namespace.c
+++ b/kernel/pid_namespace.c
@@ -370,6 +370,12 @@ static unsigned int pidns_inum(void *ns)
return pid_ns->proc_inum;
}
+static long long pidns_snum(void *ns)
+{
+ struct pid_namespace *pid_ns = ns;
+ return pid_ns->serial_num;
+}
+
const struct proc_ns_operations pidns_operations = {
.name = "pid",
.type = CLONE_NEWPID,
@@ -377,6 +383,7 @@ const struct proc_ns_operations pidns_operations = {
.put = pidns_put,
.install = pidns_install,
.inum = pidns_inum,
+ .snum = pidns_snum,
};
static __init int pid_namespaces_init(void)
diff --git a/kernel/user_namespace.c b/kernel/user_namespace.c
index 5c5c399..3f04df5 100644
--- a/kernel/user_namespace.c
+++ b/kernel/user_namespace.c
@@ -896,6 +896,12 @@ static unsigned int userns_inum(void *ns)
return user_ns->proc_inum;
}
+static long long userns_snum(void *ns)
+{
+ struct user_namespace *user_ns = ns;
+ return user_ns->serial_num;
+}
+
const struct proc_ns_operations userns_operations = {
.name = "user",
.type = CLONE_NEWUSER,
@@ -903,6 +909,7 @@ const struct proc_ns_operations userns_operations = {
.put = userns_put,
.install = userns_install,
.inum = userns_inum,
+ .snum = userns_snum,
};
static __init int user_namespaces_init(void)
diff --git a/kernel/utsname.c b/kernel/utsname.c
index d0cf7b5..ffeac1b 100644
--- a/kernel/utsname.c
+++ b/kernel/utsname.c
@@ -132,6 +132,13 @@ static unsigned int utsns_inum(void *vp)
return ns->proc_inum;
}
+static long long utsns_snum(void *vp)
+{
+ struct uts_namespace *ns = vp;
+
+ return ns->serial_num;
+}
+
const struct proc_ns_operations utsns_operations = {
.name = "uts",
.type = CLONE_NEWUTS,
@@ -139,4 +146,5 @@ const struct proc_ns_operations utsns_operations = {
.put = utsns_put,
.install = utsns_install,
.inum = utsns_inum,
+ .snum = utsns_snum,
};
diff --git a/net/core/net_namespace.c b/net/core/net_namespace.c
index 3b5cfdb..c402eea 100644
--- a/net/core/net_namespace.c
+++ b/net/core/net_namespace.c
@@ -671,6 +671,12 @@ static unsigned int netns_inum(void *ns)
return net->proc_inum;
}
+static long long netns_snum(void *ns)
+{
+ struct net *net = ns;
+ return net->serial_num;
+}
+
const struct proc_ns_operations netns_operations = {
.name = "net",
.type = CLONE_NEWNET,
@@ -678,5 +684,6 @@ const struct proc_ns_operations netns_operations = {
.put = netns_put,
.install = netns_install,
.inum = netns_inum,
+ .snum = netns_snum,
};
#endif
--
1.7.1
^ permalink raw reply related
* [PATCH V4 1/8] namespaces: assign each namespace instance a serial number
From: Richard Guy Briggs @ 2014-08-21 1:09 UTC (permalink / raw)
To: linux-audit-H+wXaHxf7aLQT0dZR+AlfA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-api-u79uwXL29TY76Z2rM5mHXA
Cc: ebiederm-aS9lmoZGLiVWk0Htik3J/w, eparis-H+wXaHxf7aLQT0dZR+AlfA,
sgrubb-H+wXaHxf7aLQT0dZR+AlfA
In-Reply-To: <cover.1408581429.git.rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Generate and assign a serial number per namespace instance since boot.
Use a serial number per namespace (unique across one boot of one kernel)
instead of the inode number (which is claimed to have had the right to change
reserved and is not necessarily unique if there is more than one proc fs) to
uniquely identify it per kernel boot.
Signed-off-by: Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/mount.h | 1 +
fs/namespace.c | 1 +
include/linux/ipc_namespace.h | 1 +
include/linux/nsproxy.h | 8 ++++++++
include/linux/pid_namespace.h | 1 +
include/linux/user_namespace.h | 1 +
include/linux/utsname.h | 1 +
include/net/net_namespace.h | 1 +
init/version.c | 1 +
ipc/msgutil.c | 1 +
ipc/namespace.c | 2 ++
kernel/nsproxy.c | 17 +++++++++++++++++
kernel/pid.c | 1 +
kernel/pid_namespace.c | 2 ++
kernel/user.c | 1 +
kernel/user_namespace.c | 2 ++
kernel/utsname.c | 2 ++
net/core/net_namespace.c | 8 +++++++-
18 files changed, 51 insertions(+), 1 deletions(-)
diff --git a/fs/mount.h b/fs/mount.h
index d55297f..c076f99 100644
--- a/fs/mount.h
+++ b/fs/mount.h
@@ -5,6 +5,7 @@
struct mnt_namespace {
atomic_t count;
unsigned int proc_inum;
+ long long serial_num;
struct mount * root;
struct list_head list;
struct user_namespace *user_ns;
diff --git a/fs/namespace.c b/fs/namespace.c
index 182bc41..9af49ff 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -2486,6 +2486,7 @@ static struct mnt_namespace *alloc_mnt_ns(struct user_namespace *user_ns)
kfree(new_ns);
return ERR_PTR(ret);
}
+ new_ns->serial_num = ns_serial();
new_ns->seq = atomic64_add_return(1, &mnt_ns_seq);
atomic_set(&new_ns->count, 1);
new_ns->root = NULL;
diff --git a/include/linux/ipc_namespace.h b/include/linux/ipc_namespace.h
index 35e7eca..8ccfb2d 100644
--- a/include/linux/ipc_namespace.h
+++ b/include/linux/ipc_namespace.h
@@ -69,6 +69,7 @@ struct ipc_namespace {
struct user_namespace *user_ns;
unsigned int proc_inum;
+ long long serial_num;
};
extern struct ipc_namespace init_ipc_ns;
diff --git a/include/linux/nsproxy.h b/include/linux/nsproxy.h
index b4ec59d..8e5fe0d 100644
--- a/include/linux/nsproxy.h
+++ b/include/linux/nsproxy.h
@@ -66,6 +66,14 @@ static inline struct nsproxy *task_nsproxy(struct task_struct *tsk)
return rcu_dereference(tsk->nsproxy);
}
+long long ns_serial(void);
+enum {
+ NS_IPC_INIT_SN = 1,
+ NS_UTS_INIT_SN = 2,
+ NS_USER_INIT_SN = 3,
+ NS_PID_INIT_SN = 4,
+};
+
int copy_namespaces(unsigned long flags, struct task_struct *tsk);
void exit_task_namespaces(struct task_struct *tsk);
void switch_task_namespaces(struct task_struct *tsk, struct nsproxy *new);
diff --git a/include/linux/pid_namespace.h b/include/linux/pid_namespace.h
index 7246ef3..4d8023e 100644
--- a/include/linux/pid_namespace.h
+++ b/include/linux/pid_namespace.h
@@ -43,6 +43,7 @@ struct pid_namespace {
int hide_pid;
int reboot; /* group exit code if this pidns was rebooted */
unsigned int proc_inum;
+ long long serial_num;
};
extern struct pid_namespace init_pid_ns;
diff --git a/include/linux/user_namespace.h b/include/linux/user_namespace.h
index 4836ba3..159ac26 100644
--- a/include/linux/user_namespace.h
+++ b/include/linux/user_namespace.h
@@ -27,6 +27,7 @@ struct user_namespace {
kuid_t owner;
kgid_t group;
unsigned int proc_inum;
+ long long serial_num;
/* Register of per-UID persistent keyrings for this namespace */
#ifdef CONFIG_PERSISTENT_KEYRINGS
diff --git a/include/linux/utsname.h b/include/linux/utsname.h
index 239e277..8490197 100644
--- a/include/linux/utsname.h
+++ b/include/linux/utsname.h
@@ -24,6 +24,7 @@ struct uts_namespace {
struct new_utsname name;
struct user_namespace *user_ns;
unsigned int proc_inum;
+ long long serial_num;
};
extern struct uts_namespace init_uts_ns;
diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h
index 361d260..5238a06 100644
--- a/include/net/net_namespace.h
+++ b/include/net/net_namespace.h
@@ -61,6 +61,7 @@ struct net {
struct user_namespace *user_ns; /* Owning user namespace */
unsigned int proc_inum;
+ long long serial_num;
struct proc_dir_entry *proc_net;
struct proc_dir_entry *proc_net_stat;
diff --git a/init/version.c b/init/version.c
index 1a4718e..cfdcb85 100644
--- a/init/version.c
+++ b/init/version.c
@@ -36,6 +36,7 @@ struct uts_namespace init_uts_ns = {
},
.user_ns = &init_user_ns,
.proc_inum = PROC_UTS_INIT_INO,
+ .serial_num = NS_UTS_INIT_SN /* ns_serial() */,
};
EXPORT_SYMBOL_GPL(init_uts_ns);
diff --git a/ipc/msgutil.c b/ipc/msgutil.c
index 7e70959..9aa66ae 100644
--- a/ipc/msgutil.c
+++ b/ipc/msgutil.c
@@ -32,6 +32,7 @@ struct ipc_namespace init_ipc_ns = {
.count = ATOMIC_INIT(1),
.user_ns = &init_user_ns,
.proc_inum = PROC_IPC_INIT_INO,
+ .serial_num = NS_IPC_INIT_SN /* ns_serial() */,
};
atomic_t nr_ipc_ns = ATOMIC_INIT(1);
diff --git a/ipc/namespace.c b/ipc/namespace.c
index 59451c1..76dac5c 100644
--- a/ipc/namespace.c
+++ b/ipc/namespace.c
@@ -41,6 +41,8 @@ static struct ipc_namespace *create_ipc_ns(struct user_namespace *user_ns,
}
atomic_inc(&nr_ipc_ns);
+ ns->serial_num = ns_serial();
+
sem_init_ns(ns);
msg_init_ns(ns);
shm_init_ns(ns);
diff --git a/kernel/nsproxy.c b/kernel/nsproxy.c
index 8e78110..93cb380 100644
--- a/kernel/nsproxy.c
+++ b/kernel/nsproxy.c
@@ -41,6 +41,23 @@ struct nsproxy init_nsproxy = {
#endif
};
+/**
+ * ns_serial - compute a serial number for the namespace
+ *
+ * Compute a serial number for the namespace to uniquely identify it in
+ * audit records.
+ */
+long long ns_serial(void)
+{
+ static atomic64_t serial = ATOMIC_INIT(4); /* reserved for IPC, UTS, user, PID */
+ long long ret;
+
+ ret = atomic64_add_return(1, &serial);
+ BUG_ON(!ret);
+
+ return ret;
+}
+
static inline struct nsproxy *create_nsproxy(void)
{
struct nsproxy *nsproxy;
diff --git a/kernel/pid.c b/kernel/pid.c
index 9b9a266..3bf7127 100644
--- a/kernel/pid.c
+++ b/kernel/pid.c
@@ -80,6 +80,7 @@ struct pid_namespace init_pid_ns = {
.child_reaper = &init_task,
.user_ns = &init_user_ns,
.proc_inum = PROC_PID_INIT_INO,
+ .serial_num = NS_PID_INIT_SN /* ns_serial() */,
};
EXPORT_SYMBOL_GPL(init_pid_ns);
diff --git a/kernel/pid_namespace.c b/kernel/pid_namespace.c
index db95d8e..40a8b36 100644
--- a/kernel/pid_namespace.c
+++ b/kernel/pid_namespace.c
@@ -109,6 +109,8 @@ static struct pid_namespace *create_pid_namespace(struct user_namespace *user_ns
if (err)
goto out_free_map;
+ ns->serial_num = ns_serial();
+
kref_init(&ns->kref);
ns->level = level;
ns->parent = get_pid_ns(parent_pid_ns);
diff --git a/kernel/user.c b/kernel/user.c
index 4efa393..2f597e0 100644
--- a/kernel/user.c
+++ b/kernel/user.c
@@ -51,6 +51,7 @@ struct user_namespace init_user_ns = {
.owner = GLOBAL_ROOT_UID,
.group = GLOBAL_ROOT_GID,
.proc_inum = PROC_USER_INIT_INO,
+ .serial_num = NS_USER_INIT_SN /* ns_serial() */,
#ifdef CONFIG_PERSISTENT_KEYRINGS
.persistent_keyring_register_sem =
__RWSEM_INITIALIZER(init_user_ns.persistent_keyring_register_sem),
diff --git a/kernel/user_namespace.c b/kernel/user_namespace.c
index fcc0256..5c5c399 100644
--- a/kernel/user_namespace.c
+++ b/kernel/user_namespace.c
@@ -92,6 +92,8 @@ int create_user_ns(struct cred *new)
return ret;
}
+ ns->serial_num = ns_serial();
+
atomic_set(&ns->count, 1);
/* Leave the new->user_ns reference with the new user namespace. */
ns->parent = parent_ns;
diff --git a/kernel/utsname.c b/kernel/utsname.c
index fd39312..d0cf7b5 100644
--- a/kernel/utsname.c
+++ b/kernel/utsname.c
@@ -48,6 +48,8 @@ static struct uts_namespace *clone_uts_ns(struct user_namespace *user_ns,
return ERR_PTR(err);
}
+ ns->serial_num = ns_serial();
+
down_read(&uts_sem);
memcpy(&ns->name, &old_ns->name, sizeof(ns->name));
ns->user_ns = get_user_ns(user_ns);
diff --git a/net/core/net_namespace.c b/net/core/net_namespace.c
index 85b6269..3b5cfdb 100644
--- a/net/core/net_namespace.c
+++ b/net/core/net_namespace.c
@@ -384,7 +384,13 @@ EXPORT_SYMBOL_GPL(get_net_ns_by_pid);
static __net_init int net_ns_net_init(struct net *net)
{
- return proc_alloc_inum(&net->proc_inum);
+ int ret;
+
+ ret = proc_alloc_inum(&net->proc_inum);
+ if (ret)
+ return ret;
+ net->serial_num = ns_serial();
+ return 0;
}
static __net_exit void net_ns_net_exit(struct net *net)
--
1.7.1
^ permalink raw reply related
* [PATCH V4 0/8] namespaces: log namespaces per task
From: Richard Guy Briggs @ 2014-08-21 1:09 UTC (permalink / raw)
To: linux-audit-H+wXaHxf7aLQT0dZR+AlfA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-api-u79uwXL29TY76Z2rM5mHXA
Cc: ebiederm-aS9lmoZGLiVWk0Htik3J/w, eparis-H+wXaHxf7aLQT0dZR+AlfA,
sgrubb-H+wXaHxf7aLQT0dZR+AlfA
The purpose is to track namespace instances in use by logged processes from the
perspective of init_*_ns by assigning each a per-kernel, per-boot serial
number.
1/8 defines a function to generate them and assigns them.
Use a serial number per namespace (unique across one boot of one kernel)
instead of the inode number (which is claimed to have had the right to change
reserved and is not necessarily unique if there is more than one proc fs). It
could be argued that the inode numbers have now become a defacto interface and
can't change now, but I'm proposing this approach to see if this helps address
some of the objections to the earlier patchset.
2/8 adds access functions to get to the serial numbers in a similar way to
inode access for namespace proc operations.
3/8 implements, as suggested by Serge Hallyn, making these serial numbers
available in /proc/self/ns/{ipc,mnt,net,pid,user,uts}_snum. I chose "snum"
instead of "seq" for consistency with inum and there are a number of other uses
of "seq" in the namespace code.
4/8 Document proc's ns entries structure in Documentation/filesystems/proc.txt
5/8 exposes proc's ns entries structure which lists a number of useful
operations per namespace type for other subsystems to use.
6/8 provides an example of usage for audit_log_task_info() which is used by
syscall audits, among others. audit_log_task() and audit_common_recv_message()
would be other potential use cases.
Proposed output format:
This differs slightly from Aristeu's patch because of the label conflict with
"pid=" due to including it in existing records rather than it being a seperate
record. It has now returned to being a seperate record. The serial numbers
are printed in hex.
type=NS_INFO msg=audit(1408577535.306:82): netns=8 utsns=2 ipcns=1 pidns=4 userns=3 mntns=5
7/8 tracks the creation and deletion of of namespaces, listing the type of
namespace instance, related namespace id if there is one and the newly minted
serial number.
Proposed output format for initial namespace creation:
type=AUDIT_NS_INIT_UTS msg=audit(1408577534.868:5): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel old_utsns=0 utsns=2 res=1
type=AUDIT_NS_INIT_USER msg=audit(1408577534.868:6): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel old_userns=0 userns=3 res=1
type=AUDIT_NS_INIT_PID msg=audit(1408577534.868:7): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel old_pidns=0 pidns=4 res=1
type=AUDIT_NS_INIT_MNT msg=audit(1408577534.868:8): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel old_mntns=0 mntns=5 res=1
type=AUDIT_NS_INIT_IPC msg=audit(1408577534.868:9): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel old_ipcns=0 ipcns=1 res=1
type=AUDIT_NS_INIT_NET msg=audit(1408577533.500:10): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel old_netns=0 netns=7 res=1
And a CLONE action would result in:
type=type=AUDIT_NS_INIT_NET msg=audit(1408577535.306:81): pid=481 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 old_netns=7 netns=8 res=1
type=type=AUDIT_NS_INIT_MNT msg=audit(1408577535.307:83): pid=481 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 old_mntns=5 mntns=9 res=1
While deleting a namespace would result in:
type=type=AUDIT_NS_DEL_MNT msg=audit(1408577552.221:85): pid=481 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 mntns=9 res=1
8/8 change audit startup from __initcall to subsys_initcall to get it started
earlier to be able to receive initial namespace log messages.
v3 -> v4:
Seperate out the NS_INFO message from the SYSCALL message.
Moved audit_log_namespace_info() out of audit_log_task_info().
Use a seperate message type per namespace type for each of INIT/DEL.
Make ns= easier to search across NS_INFO and NS_INIT/DEL_XXX msg types.
Add /proc/<pid>/ns/ documentation.
Fix dynamic initial ns logging.
v2 -> v3:
Use atomic64_t in ns_serial to simplify it.
Avoid funciton duplication in proc, keying on dentry.
Squash down audit patch to avoid rcu sleep issues.
Add tracking for creation and deletion of namespace instances.
v1 -> v2:
Avoid rollover by switching from an int to a long long.
Change rollover behaviour from simply avoiding zero to raising a BUG.
Expose serial numbers in /proc/<pid>/ns/*_snum.
Expose ns_entries and use it in audit.
Notes:
As for CAP_AUDIT_READ, a patchset has been accepted upstream to check
capabilities of userspace processes that try to join netlink broadcast groups.
This set does not try to solve the non-init namespace audit messages and
auditd problem yet. That will come later, likely with additional auditd
instances running in another namespace with a limited ability to influence the
master auditd. I echo Eric B's idea that messages destined for different
namespaces would have to be tailored for that namespace with references that
make sense (such as the right pid number reported to that pid namespace, and
not leaking info about parents or peers).
Questions:
Is there a way to link serial numbers of namespaces involved in migration of a
container to another kernel? It sounds like what is needed is a part of a
mangement application that is able to pull the audit records from constituent
hosts to build an audit trail of a container.
What additional events should list this information?
Does this present any problematic information leaks? Only CAP_AUDIT_CONTROL
(and now CAP_AUDIT_READ) in init_user_ns can get to this information in
the init namespace at the moment from audit. *However*, the addition of the
proc/<pid>/ns/*_snum does make it available to other processes now.
Richard Guy Briggs (8):
namespaces: assign each namespace instance a serial number
namespaces: expose namespace instance serial number in proc_ns_operations
namespaces: expose ns instance serial numbers in proc
Documentation: add a section for /proc/<pid>/ns/
namespaces: expose ns_entries
audit: log namespace serial numbers
audit: log creation and deletion of namespace instances
audit: initialize at subsystem time rather than device time
Documentation/filesystems/proc.txt | 16 +++++++
fs/mount.h | 1 +
fs/namespace.c | 20 +++++++++
fs/proc/namespaces.c | 35 ++++++++++++----
include/linux/audit.h | 15 +++++++
include/linux/ipc_namespace.h | 1 +
include/linux/nsproxy.h | 8 ++++
include/linux/pid_namespace.h | 1 +
include/linux/proc_ns.h | 2 +
include/linux/user_namespace.h | 1 +
include/linux/utsname.h | 1 +
include/net/net_namespace.h | 1 +
include/uapi/linux/audit.h | 13 ++++++
init/version.c | 1 +
ipc/msgutil.c | 1 +
ipc/namespace.c | 20 +++++++++
kernel/audit.c | 78 +++++++++++++++++++++++++++++++++++-
kernel/auditsc.c | 2 +
kernel/nsproxy.c | 17 ++++++++
kernel/pid.c | 1 +
kernel/pid_namespace.c | 19 +++++++++
kernel/user.c | 1 +
kernel/user_namespace.c | 20 +++++++++
kernel/utsname.c | 21 ++++++++++
net/core/net_namespace.c | 27 ++++++++++++-
security/integrity/ima/ima_api.c | 2 +
26 files changed, 314 insertions(+), 11 deletions(-)
^ permalink raw reply
* Re: [PATCH V3 0/6] namespaces: log namespaces per task
From: Nicolas Dichtel @ 2014-08-20 17:43 UTC (permalink / raw)
To: Richard Guy Briggs, Eric W. Biederman
Cc: linux-api-u79uwXL29TY76Z2rM5mHXA,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-audit-H+wXaHxf7aLQT0dZR+AlfA, netdev
In-Reply-To: <20140820162511.GS4462-bcJWsdo4jJjeVoXN4CMphl7TgLCtbB0G@public.gmane.org>
Le 20/08/2014 18:25, Richard Guy Briggs a écrit :
> On 14/08/19, Eric W. Biederman wrote:
>> Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:
>>
>>> On 14/05/20, Richard Guy Briggs wrote:
>>>> On 14/05/20, Eric Paris wrote:
>>>>> On Tue, 2014-05-20 at 09:12 -0400, Richard Guy Briggs wrote:
>>>>>> The purpose is to track namespaces in use by logged processes from the
>>>>>> perspective of init_*_ns.
>>>
>>> (Including the Linux API list due to the additions to /proc/<pid>/ns/.
>>> Please see http://www.kernelhub.org/?p=2&msg=477668 and in particular
>>> http://www.kernelhub.org/?msg=477678&p=2 )
>>
>> Sigh if you have to use something like this use the proc inode
>> number. It is the same thing.
>>
>> I hate to claim it is unique absent of the proc superblock but it is and
>> will be for the forseable future.
>>
>> It would be better to include the block device number that appears in
>> proc of 3h of the primary mount of to qualify the number. But it is not
>> particularly important. Coming up with an additional unique number that
>> needs to be maintained seems stronlgy silly.
>
> I am reading a contradiction here:
> https://www.redhat.com/archives/linux-audit/2013-March/msg00032.html
>
> and this posting went completely ignored:
> https://www.redhat.com/archives/linux-audit/2014-January/msg00180.html
>
> And then there was this patchset and thread where there was some good
> discussion to clarify the use case:
> https://lkml.org/lkml/2014/4/22/662
>
> Then V2:
> https://lkml.org/lkml/2014/5/9/637
>
> Then V3 3 months ago:
> https://www.redhat.com/archives/linux-audit/2014-May/msg00071.html
>
> I'm about to post another version of the patchset addressing Eric Paris'
> concerns about record types, field naming...
I also try to find a solution to identify netns in userland to solve
some network problems (see
http://thread.gmane.org/gmane.linux.network/315933/focus=321753).
This serial number solution may be reused for this.
We really need to find a way to solve this.
Regards,
Nicolas
^ permalink raw reply
* Re: [PATCH V3 0/6] namespaces: log namespaces per task
From: Richard Guy Briggs @ 2014-08-20 16:25 UTC (permalink / raw)
To: Eric W. Biederman
Cc: linux-api-u79uwXL29TY76Z2rM5mHXA,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Eric Paris,
linux-audit-H+wXaHxf7aLQT0dZR+AlfA, Steve Grubb
In-Reply-To: <8738crst5i.fsf-JOvCrm2gF+uungPnsOpG7nhyD016LWXt@public.gmane.org>
On 14/08/19, Eric W. Biederman wrote:
> Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:
>
> > On 14/05/20, Richard Guy Briggs wrote:
> >> On 14/05/20, Eric Paris wrote:
> >> > On Tue, 2014-05-20 at 09:12 -0400, Richard Guy Briggs wrote:
> >> > > The purpose is to track namespaces in use by logged processes from the
> >> > > perspective of init_*_ns.
> >
> > (Including the Linux API list due to the additions to /proc/<pid>/ns/.
> > Please see http://www.kernelhub.org/?p=2&msg=477668 and in particular
> > http://www.kernelhub.org/?msg=477678&p=2 )
>
> Sigh if you have to use something like this use the proc inode
> number. It is the same thing.
>
> I hate to claim it is unique absent of the proc superblock but it is and
> will be for the forseable future.
>
> It would be better to include the block device number that appears in
> proc of 3h of the primary mount of to qualify the number. But it is not
> particularly important. Coming up with an additional unique number that
> needs to be maintained seems stronlgy silly.
I am reading a contradiction here:
https://www.redhat.com/archives/linux-audit/2013-March/msg00032.html
and this posting went completely ignored:
https://www.redhat.com/archives/linux-audit/2014-January/msg00180.html
And then there was this patchset and thread where there was some good
discussion to clarify the use case:
https://lkml.org/lkml/2014/4/22/662
Then V2:
https://lkml.org/lkml/2014/5/9/637
Then V3 3 months ago:
https://www.redhat.com/archives/linux-audit/2014-May/msg00071.html
I'm about to post another version of the patchset addressing Eric Paris'
concerns about record types, field naming...
> Eric
>
> > <snip>
> >
> >> > > 5/6 provides an example of usage for audit_log_task_info() which is used by
> >> > > syscall audits, among others. audit_log_task() and audit_common_recv_message()
> >> > > would be other potential use cases.
> >> > >
> >> > > Proposed output format:
> >> > > This differs slightly from Aristeu's patch because of the label conflict with
> >> > > "pid=" due to including it in existing records rather than it being a seperate
> >> > > record. The serial numbers are printed in hex.
> >> > > type=SYSCALL msg=audit(1399651071.433:72): arch=c000003e syscall=272 success=yes exit=0 a0=40000000 a1=ffffffffffffffff a2=0 a3=22 items=0 ppid=1 pid=483 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="(t-daemon)" exe="/usr/lib/systemd/systemd" netns=97 utsns=2 ipcns=1 pidns=4 userns=3 mntns=5 subj=system_u:system_r:init_t:s0 key=(null)
> >> >
> >> > I'm undecided if I'd rather see this as a separate NS_INFO record type.
> >> > It would mean we could filter them out of the logs...
> >>
> >> I don't have a strong opinion either way. Steve G.'s opinion would be
> >> helpful here.
> >
> > Breaking this out into a seperate record type would mean calling
> > audit_log_namespace_info() from the callers of audit_log_task_info()
> > (presently audit_log_link_denied(), ima_audit_measurement(),
> > audit_log_exit() ), but would make it easier to emit with other record
> > types.
> >
> >> > Do we print out lots of pidns=0 for tasks not in a newly created NS? Do
> >> > we want to?
> >>
> >> There is no "pidns=0", but I understand your point. This would come
> >> back to Steve G.'s point about disappearing fields, and the value of
> >> having it as a seperate record that could be filtered.
> >>
> >> > > 6/6 tracks the creation and deletion of of namespaces, listing the type of
> >> > > namespace instance, related namespace id if there is one and the newly minted
> >> > > serial number.
> >> > >
> >> > > Proposed output format:
> >> > > type=NS_INIT msg=audit(1400217435.706:94): pid=524 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:mount_t:s0 type=20000 old_snum=0 snum=a1 res=1
> >> >
> >> > I'd love to be able to grep for netns=20 and find both the NS_INIT and
> >> > the SYSCALL/NS_INFO records, instead of having them named different
> >> > things. So basically I think you want to translate the type= into a
> >> > string for the old_X= and X=...
> >>
> >> That actually makes a bit more sense, and we could do away with the
> >> "type=" field since the "Xns=" fields are self-describing.
> >
> > Steve, how do you feel about this since the NS_INIT/NS_DEL record type
> > would have changing fields amongst the 6 namespace types. Only one of
> > the 6 would be present in each message. I suppose we could have an
> > NS_INIT_XXX/NS_DEL_XXX for each namespace type.
> >
> >> - RGB
> >
> > - RGB
> >
> > --
> > Richard Guy Briggs <rbriggs-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> > Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat
> > Remote, Ottawa, Canada
> > Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
- RGB
--
Richard Guy Briggs <rbriggs-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat
Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
^ permalink raw reply
* Re: [PATCH V3 0/6] namespaces: log namespaces per task
From: Eric W. Biederman @ 2014-08-20 4:04 UTC (permalink / raw)
To: Richard Guy Briggs
Cc: Eric Paris, Steve Grubb,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-audit-H+wXaHxf7aLQT0dZR+AlfA, serge-A9i7LUbDfNHQT0dZR+AlfA,
linux-api-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20140819164617.GE9003-bcJWsdo4jJjeVoXN4CMphl7TgLCtbB0G@public.gmane.org>
Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:
> On 14/05/20, Richard Guy Briggs wrote:
>> On 14/05/20, Eric Paris wrote:
>> > On Tue, 2014-05-20 at 09:12 -0400, Richard Guy Briggs wrote:
>> > > The purpose is to track namespaces in use by logged processes from the
>> > > perspective of init_*_ns.
>
> (Including the Linux API list due to the additions to /proc/<pid>/ns/.
> Please see http://www.kernelhub.org/?p=2&msg=477668 and in particular
> http://www.kernelhub.org/?msg=477678&p=2 )
Sigh if you have to use something like this use the proc inode
number. It is the same thing.
I hate to claim it is unique absent of the proc superblock but it is and
will be for the forseable future.
It would be better to include the block device number that appears in
proc of 3h of the primary mount of to qualify the number. But it is not
particularly important. Coming up with an additional unique number that
needs to be maintained seems stronlgy silly.
Eric
> <snip>
>
>> > > 5/6 provides an example of usage for audit_log_task_info() which is used by
>> > > syscall audits, among others. audit_log_task() and audit_common_recv_message()
>> > > would be other potential use cases.
>> > >
>> > > Proposed output format:
>> > > This differs slightly from Aristeu's patch because of the label conflict with
>> > > "pid=" due to including it in existing records rather than it being a seperate
>> > > record. The serial numbers are printed in hex.
>> > > type=SYSCALL msg=audit(1399651071.433:72): arch=c000003e syscall=272 success=yes exit=0 a0=40000000 a1=ffffffffffffffff a2=0 a3=22 items=0 ppid=1 pid=483 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="(t-daemon)" exe="/usr/lib/systemd/systemd" netns=97 utsns=2 ipcns=1 pidns=4 userns=3 mntns=5 subj=system_u:system_r:init_t:s0 key=(null)
>> >
>> > I'm undecided if I'd rather see this as a separate NS_INFO record type.
>> > It would mean we could filter them out of the logs...
>>
>> I don't have a strong opinion either way. Steve G.'s opinion would be
>> helpful here.
>
> Breaking this out into a seperate record type would mean calling
> audit_log_namespace_info() from the callers of audit_log_task_info()
> (presently audit_log_link_denied(), ima_audit_measurement(),
> audit_log_exit() ), but would make it easier to emit with other record
> types.
>
>> > Do we print out lots of pidns=0 for tasks not in a newly created NS? Do
>> > we want to?
>>
>> There is no "pidns=0", but I understand your point. This would come
>> back to Steve G.'s point about disappearing fields, and the value of
>> having it as a seperate record that could be filtered.
>>
>> > > 6/6 tracks the creation and deletion of of namespaces, listing the type of
>> > > namespace instance, related namespace id if there is one and the newly minted
>> > > serial number.
>> > >
>> > > Proposed output format:
>> > > type=NS_INIT msg=audit(1400217435.706:94): pid=524 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:mount_t:s0 type=20000 old_snum=0 snum=a1 res=1
>> >
>> > I'd love to be able to grep for netns=20 and find both the NS_INIT and
>> > the SYSCALL/NS_INFO records, instead of having them named different
>> > things. So basically I think you want to translate the type= into a
>> > string for the old_X= and X=...
>>
>> That actually makes a bit more sense, and we could do away with the
>> "type=" field since the "Xns=" fields are self-describing.
>
> Steve, how do you feel about this since the NS_INIT/NS_DEL record type
> would have changing fields amongst the 6 namespace types. Only one of
> the 6 would be present in each message. I suppose we could have an
> NS_INIT_XXX/NS_DEL_XXX for each namespace type.
>
>> - RGB
>
> - RGB
>
> --
> Richard Guy Briggs <rbriggs-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat
> Remote, Ottawa, Canada
> Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
^ permalink raw reply
* Re: [PATCH RFC v4 net-next 17/26] tracing: allow eBPF programs to be attached to events
From: Alexei Starovoitov @ 2014-08-19 18:39 UTC (permalink / raw)
To: Andy Lutomirski
Cc: Daniel Borkmann, H. Peter Anvin, Andrew Morton, Linux API,
Chema Gonzalez, Eric Dumazet, David S. Miller, Brendan Gregg,
Linus Torvalds, Steven Rostedt, LKML, Peter Zijlstra, Kees Cook,
Network Development, Ingo Molnar
In-Reply-To: <CALCETrUqop+UB-BhyX4Y41kELO+6kcFdS1F7ZyN0CzRwg4UGhA@mail.gmail.com>
On Fri, Aug 15, 2014 at 12:18 PM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Fri, Aug 15, 2014 at 12:16 PM, Alexei Starovoitov <ast@plumgrid.com> wrote:
>> This bpf_context struct for tracing is trying to answer the question:
>> 'what's the most convenient way to access tracepoint arguments
>> from a script'.
>> When kernel code has something like:
>> trace_kfree_skb(skb, net_tx_action);
>> the script needs to be able to access this 'skb' and 'net_tx_action'
>> values through _single_ data structure.
>> In this proposal they are ctx->arg1 and ctx->arg2.
>> I've considered having different bpf_context's for every event, but
>> the complexity explodes. I need to hack all event definitions and so on.
>> imo it's better to move complexity to userspace, so program author
>> or high level language abstracts these details.
>
> I still don't understand why making them long instead of u64 is
> helpful, though. I feel like I'm missing obvious here.
The problem statement:
- tracepoint events are defined as:
TRACE_EVENT(sock_exceed_buf_limit,
TP_PROTO(struct sock *sk, struct proto *prot, long allocated),
TP_ARGS(sk, prot, allocated),
- from eBPF C program (or higher level language) I would like
to access these tracepoint arguments as
ctx->arg1, ctx->arg2, ctx->arg3
- accessing tracepoint fields after buffer copy is not an option,
since it's unnecessary alloc/copy/free of a lot of values
(including strings) per event that programs will mostly ignore.
If needed, programs can fetch them on demand.
Bad approach #1
- define different bpf_context per event and customize eBPF verifier
to have different program types per event, so particular program
can be attached to one particular event only
Cons: quite complex, require trace/events/*.h hacking,
one ebpf program cannot be used to attach to multiple events
So #1 is no go.
Approach #2
- define bpf_context once for all tracepoint events as:
struct bpf_context {
unsigned long arg1, arg2, arg3, ...
};
and main ftrace.h macro as:
struct bpf_context ctx;
populate_bpf_context(&ctx, args, 0, 0, 0, 0, 0);
trace_filter_call_bpf(ftrace_file->filter, &ctx);
where 'args' is a macro taken from TP_ARGS above and
/* called from ftrace_raw_event_*() to copy args */
void populate_bpf_context(struct bpf_context *ctx, ...)
{
va_start(args, ctx);
ctx->arg1 = va_arg(args, unsigned long);
ctx->arg2 = va_arg(args, unsigned long);
this approach relies on type promotion when args are passed
into vararg function.
On 64-bit arch our tracepoint arguments 'sk, prot, allocated' will
get stored into arg1, arg2, arg3 and the rest of argX will be zeros.
On 32-bit 'u64' types will be passed in two 'long' slots of vararg.
Obviously changing 'long' to 'u64' in bpf_context and in
populate_bpf_context() will not work, because types
are promoted to 'long'.
Disadvantage of this approach is that 32 vs 64 bit archs need to
deal with argX differently.
That's what you saw in this patch.
New approach #3
just discovered __MAPx() macro used by syscalls.h which can
be massaged for this use case, so define:
struct bpf_context {
u64 arg1, arg2, arg3,...
};
and argument casting macro as:
#define __BPF_CAST1(a,...) __CAST_TO_U64(a)
#define __BPF_CAST2(a,...) __CAST_TO_U64(a), __BPF_CAST1(__VA_ARGS__)
..
#define __BPF_CAST(a,...) __CAST_TO_U64(a), __BPF_CAST6(__VA_ARGS__)
so main ftrace.h macro becomes:
struct bpf_context __ctx = ((struct bpf_context) {
__BPF_CAST(args, 0, 0, 0, 0, 0, 0)
});
trace_filter_call_bpf(ftrace_file->filter, &__ctx);
where 'args' is still the same 'sk, prot, allocated' from TP_ARGS.
Casting macro will cast 'sk' to u64 and will assign into arg1,
'prot' will be casted to u64 and assigned to arg2, etc
All good, but the tricky part is how to cast all arguments passed
into tracepoint events into u64 without warnings on
32-bit and 64-bit architectures.
The following:
#define __CAST_TO_U64(expr) (u64) expr
will not work, since compiler will be spewing warnings about
casting pointer to integer...
The following
#define __CAST_TO_U64(expr) (u64) (long) expr
will work fine on 64-bit architectures, since all integer and
pointer types will be warning-free casted and stored
in arg1, arg2, arg3, ...
but it won't work on 32-bit architectures, since full u64
tracepoint arguments will be chopped to 'long'.
It took a lot of macro wizardry to come up with the following:
/* cast any interger or pointer type to u64 without warnings */
#define __CAST_TO_U64(expr) \
__builtin_choose_expr(sizeof(long) < sizeof(expr), \
(u64) (expr - ((typeof(expr))0)), \
(u64) (long) expr)
the tricky part is that GCC syntax-parses and warns
in both sides of __builtin_choose_expr(), so u64 case in 32-bit
archs needs to be fancy, so all types can go through it
warning free.
Though it's tricky the end result is nice.
The disadvantages of approach #2 are solved and tracepoint
arguments are stored into 'u64 argX' on both 32 and 64-bit archs.
The extra benefit is that this casting macro is way faster than
vararg approach #2.
So in V5 of this series I'm planning to use this new approach
unless there are better ideas.
full diff of this approach:
https://git.kernel.org/cgit/linux/kernel/git/ast/bpf.git/commit/?id=235d87dd7afd8d5262556cba7882d6efb25d8305
^ permalink raw reply
* Re: [PATCH V3 0/6] namespaces: log namespaces per task
From: Richard Guy Briggs @ 2014-08-19 16:46 UTC (permalink / raw)
To: Eric Paris, Steve Grubb
Cc: linux-api-u79uwXL29TY76Z2rM5mHXA,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-audit-H+wXaHxf7aLQT0dZR+AlfA,
ebiederm-aS9lmoZGLiVWk0Htik3J/w
In-Reply-To: <20140520140146.GA8079-bcJWsdo4jJjeVoXN4CMphl7TgLCtbB0G@public.gmane.org>
On 14/05/20, Richard Guy Briggs wrote:
> On 14/05/20, Eric Paris wrote:
> > On Tue, 2014-05-20 at 09:12 -0400, Richard Guy Briggs wrote:
> > > The purpose is to track namespaces in use by logged processes from the
> > > perspective of init_*_ns.
(Including the Linux API list due to the additions to /proc/<pid>/ns/.
Please see http://www.kernelhub.org/?p=2&msg=477668 and in particular
http://www.kernelhub.org/?msg=477678&p=2 )
<snip>
> > > 5/6 provides an example of usage for audit_log_task_info() which is used by
> > > syscall audits, among others. audit_log_task() and audit_common_recv_message()
> > > would be other potential use cases.
> > >
> > > Proposed output format:
> > > This differs slightly from Aristeu's patch because of the label conflict with
> > > "pid=" due to including it in existing records rather than it being a seperate
> > > record. The serial numbers are printed in hex.
> > > type=SYSCALL msg=audit(1399651071.433:72): arch=c000003e syscall=272 success=yes exit=0 a0=40000000 a1=ffffffffffffffff a2=0 a3=22 items=0 ppid=1 pid=483 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="(t-daemon)" exe="/usr/lib/systemd/systemd" netns=97 utsns=2 ipcns=1 pidns=4 userns=3 mntns=5 subj=system_u:system_r:init_t:s0 key=(null)
> >
> > I'm undecided if I'd rather see this as a separate NS_INFO record type.
> > It would mean we could filter them out of the logs...
>
> I don't have a strong opinion either way. Steve G.'s opinion would be
> helpful here.
Breaking this out into a seperate record type would mean calling
audit_log_namespace_info() from the callers of audit_log_task_info()
(presently audit_log_link_denied(), ima_audit_measurement(),
audit_log_exit() ), but would make it easier to emit with other record
types.
> > Do we print out lots of pidns=0 for tasks not in a newly created NS? Do
> > we want to?
>
> There is no "pidns=0", but I understand your point. This would come
> back to Steve G.'s point about disappearing fields, and the value of
> having it as a seperate record that could be filtered.
>
> > > 6/6 tracks the creation and deletion of of namespaces, listing the type of
> > > namespace instance, related namespace id if there is one and the newly minted
> > > serial number.
> > >
> > > Proposed output format:
> > > type=NS_INIT msg=audit(1400217435.706:94): pid=524 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:mount_t:s0 type=20000 old_snum=0 snum=a1 res=1
> >
> > I'd love to be able to grep for netns=20 and find both the NS_INIT and
> > the SYSCALL/NS_INFO records, instead of having them named different
> > things. So basically I think you want to translate the type= into a
> > string for the old_X= and X=...
>
> That actually makes a bit more sense, and we could do away with the
> "type=" field since the "Xns=" fields are self-describing.
Steve, how do you feel about this since the NS_INIT/NS_DEL record type
would have changing fields amongst the 6 namespace types. Only one of
the 6 would be present in each message. I suppose we could have an
NS_INIT_XXX/NS_DEL_XXX for each namespace type.
> - RGB
- RGB
--
Richard Guy Briggs <rbriggs-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat
Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
^ permalink raw reply
* Re: [PATCH v15 7/7] mm: Don't split THP page when syscall is called
From: Rik van Riel @ 2014-08-19 2:10 UTC (permalink / raw)
To: Minchan Kim, Andrew Morton
Cc: linux-kernel, linux-mm, Michael Kerrisk, linux-api, Hugh Dickins,
Johannes Weiner, KOSAKI Motohiro, Mel Gorman, Jason Evans,
zhangyanfei, Kirill A. Shutemov, Andrea Arcangeli,
Kirill A. Shutemov
In-Reply-To: <1408321076-2231-8-git-send-email-minchan@kernel.org>
On 08/17/2014 08:17 PM, Minchan Kim wrote:
> We don't need to split THP page when MADV_FREE syscall is
> called. It could be done when VM decide really frees it so
> we could avoid unnecessary THP split.
>
> Cc: Andrea Arcangeli <aarcange@redhat.com>
> Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Signed-off-by: Minchan Kim <minchan@kernel.org>
Acked-by: Rik van Riel <riel@redhat.com>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v15 2/7] x86: add pmd_[dirty|mkclean] for THP
From: Rik van Riel @ 2014-08-19 2:08 UTC (permalink / raw)
To: Minchan Kim, Andrew Morton
Cc: linux-kernel, linux-mm, Michael Kerrisk, linux-api, Hugh Dickins,
Johannes Weiner, KOSAKI Motohiro, Mel Gorman, Jason Evans,
zhangyanfei, Kirill A. Shutemov, Thomas Gleixner, Ingo Molnar,
H. Peter Anvin, x86, Kirill A. Shutemov
In-Reply-To: <1408321076-2231-3-git-send-email-minchan@kernel.org>
On 08/17/2014 08:17 PM, Minchan Kim wrote:
> MADV_FREE needs pmd_dirty and pmd_mkclean for detecting recent
> overwrite of the contents since MADV_FREE syscall is called for
> THP page.
>
> This patch adds pmd_dirty and pmd_mkclean for THP page MADV_FREE
> support.
>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: x86@kernel.org
> Acked-by: Zhang Yanfei <zhangyanfei@cn.fujitsu.com>
> Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Signed-off-by: Minchan Kim <minchan@kernel.org>
Acked-by: Rik van Riel <riel@redhat.com>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH v15 7/7] mm: Don't split THP page when syscall is called
From: Minchan Kim @ 2014-08-18 0:17 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-kernel, linux-mm, Michael Kerrisk, linux-api, Hugh Dickins,
Johannes Weiner, Rik van Riel, KOSAKI Motohiro, Mel Gorman,
Jason Evans, zhangyanfei, Kirill A. Shutemov, Minchan Kim,
Andrea Arcangeli, Kirill A. Shutemov
In-Reply-To: <1408321076-2231-1-git-send-email-minchan@kernel.org>
We don't need to split THP page when MADV_FREE syscall is
called. It could be done when VM decide really frees it so
we could avoid unnecessary THP split.
Cc: Andrea Arcangeli <aarcange@redhat.com>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Signed-off-by: Minchan Kim <minchan@kernel.org>
---
include/linux/huge_mm.h | 4 ++++
mm/huge_memory.c | 35 +++++++++++++++++++++++++++++++++++
mm/madvise.c | 21 ++++++++++++++++++++-
mm/rmap.c | 8 ++++++--
mm/vmscan.c | 28 ++++++++++++++++++----------
5 files changed, 83 insertions(+), 13 deletions(-)
diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
index 63579cb8d3dc..25a961256d9f 100644
--- a/include/linux/huge_mm.h
+++ b/include/linux/huge_mm.h
@@ -19,6 +19,9 @@ extern struct page *follow_trans_huge_pmd(struct vm_area_struct *vma,
unsigned long addr,
pmd_t *pmd,
unsigned int flags);
+extern int madvise_free_huge_pmd(struct mmu_gather *tlb,
+ struct vm_area_struct *vma,
+ pmd_t *pmd, unsigned long addr);
extern int zap_huge_pmd(struct mmu_gather *tlb,
struct vm_area_struct *vma,
pmd_t *pmd, unsigned long addr);
@@ -56,6 +59,7 @@ extern pmd_t *page_check_address_pmd(struct page *page,
unsigned long address,
enum page_check_address_pmd_flag flag,
spinlock_t **ptl);
+extern int pmd_freeable(pmd_t pmd);
#define HPAGE_PMD_ORDER (HPAGE_PMD_SHIFT-PAGE_SHIFT)
#define HPAGE_PMD_NR (1<<HPAGE_PMD_ORDER)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index d9a21d06b862..6d9faa652f54 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1384,6 +1384,36 @@ out:
return 0;
}
+int madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
+ pmd_t *pmd, unsigned long addr)
+
+{
+ spinlock_t *ptl;
+ struct mm_struct *mm = tlb->mm;
+ int ret = 1;
+
+ if (pmd_trans_huge_lock(pmd, vma, &ptl) == 1) {
+ struct page *page;
+ pmd_t orig_pmd;
+
+ orig_pmd = pmdp_get_and_clear(mm, addr, pmd);
+
+ /* No hugepage in swapcache */
+ page = pmd_page(orig_pmd);
+ VM_BUG_ON_PAGE(PageSwapCache(page), page);
+
+ orig_pmd = pmd_mkold(orig_pmd);
+ orig_pmd = pmd_mkclean(orig_pmd);
+
+ set_pmd_at(mm, addr, pmd, orig_pmd);
+ tlb_remove_pmd_tlb_entry(tlb, pmd, addr);
+ spin_unlock(ptl);
+ ret = 0;
+ }
+
+ return ret;
+}
+
int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
pmd_t *pmd, unsigned long addr)
{
@@ -1620,6 +1650,11 @@ unlock:
return NULL;
}
+int pmd_freeable(pmd_t pmd)
+{
+ return !pmd_dirty(pmd);
+}
+
static int __split_huge_page_splitting(struct page *page,
struct vm_area_struct *vma,
unsigned long address)
diff --git a/mm/madvise.c b/mm/madvise.c
index a21584235bb6..84badee5f46d 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -271,8 +271,26 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
spinlock_t *ptl;
pte_t *pte, ptent;
struct page *page;
+ unsigned long next;
+
+ next = pmd_addr_end(addr, end);
+ if (pmd_trans_huge(*pmd)) {
+ if (next - addr != HPAGE_PMD_SIZE) {
+#ifdef CONFIG_DEBUG_VM
+ if (!rwsem_is_locked(&mm->mmap_sem)) {
+ pr_err("%s: mmap_sem is unlocked! addr=0x%lx end=0x%lx vma->vm_start=0x%lx vma->vm_end=0x%lx\n",
+ __func__, addr, end,
+ vma->vm_start,
+ vma->vm_end);
+ BUG();
+ }
+#endif
+ split_huge_page_pmd(vma, addr, pmd);
+ } else if (!madvise_free_huge_pmd(tlb, vma, pmd, addr))
+ goto next;
+ /* fall through */
+ }
- split_huge_page_pmd(vma, addr, pmd);
if (pmd_trans_unstable(pmd))
return 0;
@@ -316,6 +334,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
}
arch_leave_lazy_mmu_mode();
pte_unmap_unlock(pte - 1, ptl);
+next:
cond_resched();
return 0;
}
diff --git a/mm/rmap.c b/mm/rmap.c
index 04c181133890..9c407576ff8e 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -704,9 +704,13 @@ static int page_referenced_one(struct page *page, struct vm_area_struct *vma,
referenced++;
/*
- * In this implmentation, MADV_FREE doesn't support THP free
+ * Use pmd_freeable instead of raw pmd_dirty because in some
+ * of architecture, pmd_dirty is not defined unless
+ * CONFIG_TRANSPARNTE_HUGE is enabled
*/
- dirty++;
+ if (!pmd_freeable(*pmd))
+ dirty++;
+
spin_unlock(ptl);
} else {
pte_t *pte;
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 76c2cc858323..d404da3ad1a5 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -976,17 +976,25 @@ static unsigned long shrink_page_list(struct list_head *page_list,
* Anonymous process memory has backing store?
* Try to allocate it some swap space here.
*/
- if (PageAnon(page) && !PageSwapCache(page) && !freeable) {
- if (!(sc->gfp_mask & __GFP_IO))
- goto keep_locked;
- if (!add_to_swap(page, page_list))
- goto activate_locked;
- may_enter_fs = 1;
-
- /* Adding to swap updated mapping */
- mapping = page_mapping(page);
+ if (PageAnon(page) && !PageSwapCache(page)) {
+ if (!freeable) {
+ if (!(sc->gfp_mask & __GFP_IO))
+ goto keep_locked;
+ if (!add_to_swap(page, page_list))
+ goto activate_locked;
+ may_enter_fs = 1;
+ /* Adding to swap updated mapping */
+ mapping = page_mapping(page);
+ } else {
+ if (likely(!PageTransHuge(page)))
+ goto unmap;
+ /* try_to_unmap isn't aware of THP page */
+ if (unlikely(split_huge_page_to_list(page,
+ page_list)))
+ goto keep_locked;
+ }
}
-
+unmap:
/*
* The page is mapped into the page tables of one or more
* processes. Try to unmap it here.
--
2.0.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v15 6/7] arm64: add pmd_[dirty|mkclean] for THP
From: Minchan Kim @ 2014-08-18 0:17 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-kernel, linux-mm, Michael Kerrisk, linux-api, Hugh Dickins,
Johannes Weiner, Rik van Riel, KOSAKI Motohiro, Mel Gorman,
Jason Evans, zhangyanfei, Kirill A. Shutemov, Minchan Kim,
Russell King, linux-arm-kernel, Will Deacon, Steve Capper,
Catalin Marinas
In-Reply-To: <1408321076-2231-1-git-send-email-minchan@kernel.org>
MADV_FREE needs pmd_dirty and pmd_mkclean for detecting recent
overwrite of the contents since MADV_FREE syscall is called for
THP page.
This patch adds pmd_dirty and pmd_mkclean for THP page MADV_FREE
support.
Cc: Russell King <linux@arm.linux.org.uk>
Cc: linux-arm-kernel@lists.infradead.org
Acked-by: Will Deacon <will.deacon@arm.com>
Acked-by: Steve Capper <steve.capper@linaro.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Minchan Kim <minchan@kernel.org>
---
arch/arm64/include/asm/pgtable.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index ffe1ba0506d1..efb1b2fc4d39 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -259,10 +259,12 @@ static inline pmd_t pte_pmd(pte_t pte)
#endif
#define pmd_young(pmd) pte_young(pmd_pte(pmd))
+#define pmd_dirty(pmd) pte_dirty(pmd_pte(pmd))
#define pmd_wrprotect(pmd) pte_pmd(pte_wrprotect(pmd_pte(pmd)))
#define pmd_mksplitting(pmd) pte_pmd(pte_mkspecial(pmd_pte(pmd)))
#define pmd_mkold(pmd) pte_pmd(pte_mkold(pmd_pte(pmd)))
#define pmd_mkwrite(pmd) pte_pmd(pte_mkwrite(pmd_pte(pmd)))
+#define pmd_mkclean(pmd) pte_pmd(pte_mkclean(pmd_pte(pmd)))
#define pmd_mkdirty(pmd) pte_pmd(pte_mkdirty(pmd_pte(pmd)))
#define pmd_mkyoung(pmd) pte_pmd(pte_mkyoung(pmd_pte(pmd)))
#define pmd_mknotpresent(pmd) (__pmd(pmd_val(pmd) & ~PMD_TYPE_MASK))
--
2.0.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v15 5/7] arm: add pmd_mkclean for THP
From: Minchan Kim @ 2014-08-18 0:17 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-kernel, linux-mm, Michael Kerrisk, linux-api, Hugh Dickins,
Johannes Weiner, Rik van Riel, KOSAKI Motohiro, Mel Gorman,
Jason Evans, zhangyanfei, Kirill A. Shutemov, Minchan Kim,
Catalin Marinas, Russell King, linux-arm-kernel, Will Deacon,
Steve Capper
In-Reply-To: <1408321076-2231-1-git-send-email-minchan@kernel.org>
MADV_FREE needs pmd_dirty and pmd_mkclean for detecting recent
overwrite of the contents since MADV_FREE syscall is called for
THP page.
This patch adds pmd_mkclean for THP page MADV_FREE support.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: linux-arm-kernel@lists.infradead.org
Acked-by: Will Deacon <will.deacon@arm.com>
Acked-by: Steve Capper <steve.capper@linaro.org>
Signed-off-by: Minchan Kim <minchan@kernel.org>
---
arch/arm/include/asm/pgtable-3level.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm/include/asm/pgtable-3level.h b/arch/arm/include/asm/pgtable-3level.h
index 06e0bc0f8b00..bc913a065270 100644
--- a/arch/arm/include/asm/pgtable-3level.h
+++ b/arch/arm/include/asm/pgtable-3level.h
@@ -234,6 +234,7 @@ PMD_BIT_FUNC(mkold, &= ~PMD_SECT_AF);
PMD_BIT_FUNC(mksplitting, |= L_PMD_SECT_SPLITTING);
PMD_BIT_FUNC(mkwrite, &= ~L_PMD_SECT_RDONLY);
PMD_BIT_FUNC(mkdirty, |= L_PMD_SECT_DIRTY);
+PMD_BIT_FUNC(mkclean, &= ~L_PMD_SECT_DIRTY);
PMD_BIT_FUNC(mkyoung, |= PMD_SECT_AF);
#define pmd_mkhuge(pmd) (__pmd(pmd_val(pmd) & ~PMD_TABLE_BIT))
--
2.0.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v15 4/7] powerpc: add pmd_[dirty|mkclean] for THP
From: Minchan Kim @ 2014-08-18 0:17 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-kernel, linux-mm, Michael Kerrisk, linux-api, Hugh Dickins,
Johannes Weiner, Rik van Riel, KOSAKI Motohiro, Mel Gorman,
Jason Evans, zhangyanfei, Kirill A. Shutemov, Minchan Kim,
Benjamin Herrenschmidt, Paul Mackerras, linuxppc-dev,
Aneesh Kumar K.V
In-Reply-To: <1408321076-2231-1-git-send-email-minchan@kernel.org>
MADV_FREE needs pmd_dirty and pmd_mkclean for detecting recent
overwrite of the contents since MADV_FREE syscall is called for
THP page.
This patch adds pmd_dirty and pmd_mkclean for THP page MADV_FREE
support.
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: linuxppc-dev@lists.ozlabs.org
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: Minchan Kim <minchan@kernel.org>
---
arch/powerpc/include/asm/pgtable-ppc64.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/powerpc/include/asm/pgtable-ppc64.h b/arch/powerpc/include/asm/pgtable-ppc64.h
index eb9261024f51..c9a4bbe8e179 100644
--- a/arch/powerpc/include/asm/pgtable-ppc64.h
+++ b/arch/powerpc/include/asm/pgtable-ppc64.h
@@ -468,9 +468,11 @@ static inline pte_t *pmdp_ptep(pmd_t *pmd)
#define pmd_pfn(pmd) pte_pfn(pmd_pte(pmd))
#define pmd_young(pmd) pte_young(pmd_pte(pmd))
+#define pmd_dirty(pmd) pte_dirty(pmd_pte(pmd))
#define pmd_mkold(pmd) pte_pmd(pte_mkold(pmd_pte(pmd)))
#define pmd_wrprotect(pmd) pte_pmd(pte_wrprotect(pmd_pte(pmd)))
#define pmd_mkdirty(pmd) pte_pmd(pte_mkdirty(pmd_pte(pmd)))
+#define pmd_mkclean(pmd) pte_pmd(pte_mkclean(pmd_pte(pmd)))
#define pmd_mkyoung(pmd) pte_pmd(pte_mkyoung(pmd_pte(pmd)))
#define pmd_mkwrite(pmd) pte_pmd(pte_mkwrite(pmd_pte(pmd)))
--
2.0.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v15 3/7] sparc: add pmd_[dirty|mkclean] for THP
From: Minchan Kim @ 2014-08-18 0:17 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-kernel, linux-mm, Michael Kerrisk, linux-api, Hugh Dickins,
Johannes Weiner, Rik van Riel, KOSAKI Motohiro, Mel Gorman,
Jason Evans, zhangyanfei, Kirill A. Shutemov, Minchan Kim,
sparclinux, David S. Miller
In-Reply-To: <1408321076-2231-1-git-send-email-minchan@kernel.org>
MADV_FREE needs pmd_dirty and pmd_mkclean for detecting recent
overwrite of the contents since MADV_FREE syscall is called for
THP page.
This patch adds pmd_dirty and pmd_mkclean for THP page MADV_FREE
support.
Acked-by: David S. Miller <davem@davemloft.net>
Cc: sparclinux@vger.kernel.org
Signed-off-by: Minchan Kim <minchan@kernel.org>
---
arch/sparc/include/asm/pgtable_64.h | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/arch/sparc/include/asm/pgtable_64.h b/arch/sparc/include/asm/pgtable_64.h
index 3770bf5c6e1b..b80a309d7e00 100644
--- a/arch/sparc/include/asm/pgtable_64.h
+++ b/arch/sparc/include/asm/pgtable_64.h
@@ -666,6 +666,13 @@ static inline unsigned long pmd_young(pmd_t pmd)
return pte_young(pte);
}
+static inline int pmd_dirty(pmd_t pmd)
+{
+ pte_t pte = __pte(pmd_val(pmd));
+
+ return pte_dirty(pte);
+}
+
static inline unsigned long pmd_write(pmd_t pmd)
{
pte_t pte = __pte(pmd_val(pmd));
@@ -723,6 +730,15 @@ static inline pmd_t pmd_mkdirty(pmd_t pmd)
return __pmd(pte_val(pte));
}
+static inline pmd_t pmd_mkclean(pmd_t pmd)
+{
+ pte_t pte = __pte(pmd_val(pmd));
+
+ pte = pte_mkclean(pte);
+
+ return __pmd(pte_val(pte));
+}
+
static inline pmd_t pmd_mkyoung(pmd_t pmd)
{
pte_t pte = __pte(pmd_val(pmd));
--
2.0.0
^ permalink raw reply related
* [PATCH v15 2/7] x86: add pmd_[dirty|mkclean] for THP
From: Minchan Kim @ 2014-08-18 0:17 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-kernel, linux-mm, Michael Kerrisk, linux-api, Hugh Dickins,
Johannes Weiner, Rik van Riel, KOSAKI Motohiro, Mel Gorman,
Jason Evans, zhangyanfei, Kirill A. Shutemov, Minchan Kim,
Thomas Gleixner, Ingo Molnar, H. Peter Anvin, x86,
Kirill A. Shutemov
In-Reply-To: <1408321076-2231-1-git-send-email-minchan@kernel.org>
MADV_FREE needs pmd_dirty and pmd_mkclean for detecting recent
overwrite of the contents since MADV_FREE syscall is called for
THP page.
This patch adds pmd_dirty and pmd_mkclean for THP page MADV_FREE
support.
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Acked-by: Zhang Yanfei <zhangyanfei@cn.fujitsu.com>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Signed-off-by: Minchan Kim <minchan@kernel.org>
---
arch/x86/include/asm/pgtable.h | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 0ec056012618..329865799653 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -104,6 +104,11 @@ static inline int pmd_young(pmd_t pmd)
return pmd_flags(pmd) & _PAGE_ACCESSED;
}
+static inline int pmd_dirty(pmd_t pmd)
+{
+ return pmd_flags(pmd) & _PAGE_DIRTY;
+}
+
static inline int pte_write(pte_t pte)
{
return pte_flags(pte) & _PAGE_RW;
@@ -267,6 +272,11 @@ static inline pmd_t pmd_mkold(pmd_t pmd)
return pmd_clear_flags(pmd, _PAGE_ACCESSED);
}
+static inline pmd_t pmd_mkclean(pmd_t pmd)
+{
+ return pmd_clear_flags(pmd, _PAGE_DIRTY);
+}
+
static inline pmd_t pmd_wrprotect(pmd_t pmd)
{
return pmd_clear_flags(pmd, _PAGE_RW);
--
2.0.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v15 1/7] mm: support madvise(MADV_FREE)
From: Minchan Kim @ 2014-08-18 0:17 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-kernel, linux-mm, Michael Kerrisk, linux-api, Hugh Dickins,
Johannes Weiner, Rik van Riel, KOSAKI Motohiro, Mel Gorman,
Jason Evans, zhangyanfei, Kirill A. Shutemov, Minchan Kim,
Kirill A. Shutemov
In-Reply-To: <1408321076-2231-1-git-send-email-minchan@kernel.org>
Linux doesn't have an ability to free pages lazy while other OS
already have been supported that named by madvise(MADV_FREE).
The gain is clear that kernel can discard freed pages rather than
swapping out or OOM if memory pressure happens.
Without memory pressure, freed pages would be reused by userspace
without another additional overhead(ex, page fault + allocation
+ zeroing).
How to work is following as.
When madvise syscall is called, VM clears dirty bit of ptes of
the range. If memory pressure happens, VM checks dirty bit of
page table and if it found still "clean", it means it's a
"lazyfree pages" so VM could discard the page instead of swapping out.
Once there was store operation for the page before VM peek a page
to reclaim, dirty bit is set so VM can swap out the page instead of
discarding.
Firstly, heavy users would be general allocators(ex, jemalloc,
tcmalloc and hope glibc supports it) and jemalloc/tcmalloc already
have supported the feature for other OS(ex, FreeBSD)
barrios@blaptop:~/benchmark/ebizzy$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 42
Stepping: 7
CPU MHz: 2801.000
BogoMIPS: 5581.64
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 4096K
NUMA node0 CPU(s): 0-3
ebizzy benchmark(./ebizzy -S 10 -n 512)
vanilla-jemalloc MADV_free-jemalloc
1 thread
records: 10 records: 10
avg: 7682.10 avg: 15306.10
std: 62.35(0.81%) std: 347.99(2.27%)
max: 7770.00 max: 15622.00
min: 7598.00 min: 14772.00
2 thread
records: 10 records: 10
avg: 12747.50 avg: 24171.00
std: 792.06(6.21%) std: 895.18(3.70%)
max: 13337.00 max: 26023.00
min: 10535.00 min: 23152.00
4 thread
records: 10 records: 10
avg: 16474.60 avg: 33717.90
std: 1496.45(9.08%) std: 2008.97(5.96%)
max: 17877.00 max: 35958.00
min: 12224.00 min: 29565.00
8 thread
records: 10 records: 10
avg: 16778.50 avg: 33308.10
std: 825.53(4.92%) std: 1668.30(5.01%)
max: 17543.00 max: 36010.00
min: 14576.00 min: 29577.00
16 thread
records: 10 records: 10
avg: 20614.40 avg: 35516.30
std: 602.95(2.92%) std: 1283.65(3.61%)
max: 21753.00 max: 37178.00
min: 19605.00 min: 33217.00
32 thread
records: 10 records: 10
avg: 22771.70 avg: 36018.50
std: 598.94(2.63%) std: 1046.76(2.91%)
max: 24035.00 max: 37266.00
min: 22108.00 min: 34149.00
In summary, MADV_FREE is about 2 time faster than MADV_DONTNEED.
Cc: Michael Kerrisk <mtk.manpages@gmail.com>
Cc: Linux API <linux-api@vger.kernel.org>
Cc: Hugh Dickins <hughd@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Jason Evans <je@fb.com>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Acked-by: Zhang Yanfei <zhangyanfei@cn.fujitsu.com>
Acked-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Minchan Kim <minchan@kernel.org>
---
include/linux/rmap.h | 9 ++-
include/linux/vm_event_item.h | 1 +
include/uapi/asm-generic/mman-common.h | 1 +
mm/madvise.c | 140 +++++++++++++++++++++++++++++++++
mm/rmap.c | 42 +++++++++-
mm/vmscan.c | 40 ++++++++--
mm/vmstat.c | 1 +
7 files changed, 222 insertions(+), 12 deletions(-)
diff --git a/include/linux/rmap.h b/include/linux/rmap.h
index be574506e6a9..0ba377b97a38 100644
--- a/include/linux/rmap.h
+++ b/include/linux/rmap.h
@@ -75,6 +75,7 @@ enum ttu_flags {
TTU_UNMAP = 1, /* unmap mode */
TTU_MIGRATION = 2, /* migration mode */
TTU_MUNLOCK = 4, /* munlock mode */
+ TTU_FREE = 8, /* free mode */
TTU_IGNORE_MLOCK = (1 << 8), /* ignore mlock */
TTU_IGNORE_ACCESS = (1 << 9), /* don't age */
@@ -181,7 +182,8 @@ static inline void page_dup_rmap(struct page *page)
* Called from mm/vmscan.c to handle paging out
*/
int page_referenced(struct page *, int is_locked,
- struct mem_cgroup *memcg, unsigned long *vm_flags);
+ struct mem_cgroup *memcg, unsigned long *vm_flags,
+ int *is_dirty);
#define TTU_ACTION(x) ((x) & TTU_ACTION_MASK)
@@ -260,9 +262,12 @@ int rmap_walk(struct page *page, struct rmap_walk_control *rwc);
static inline int page_referenced(struct page *page, int is_locked,
struct mem_cgroup *memcg,
- unsigned long *vm_flags)
+ unsigned long *vm_flags,
+ int *is_pte_dirty)
{
*vm_flags = 0;
+ if (is_pte_dirty)
+ *is_pte_dirty = 0;
return 0;
}
diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
index ced92345c963..e2d3fb1e9814 100644
--- a/include/linux/vm_event_item.h
+++ b/include/linux/vm_event_item.h
@@ -25,6 +25,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
FOR_ALL_ZONES(PGALLOC),
PGFREE, PGACTIVATE, PGDEACTIVATE,
PGFAULT, PGMAJFAULT,
+ PGLAZYFREED,
FOR_ALL_ZONES(PGREFILL),
FOR_ALL_ZONES(PGSTEAL_KSWAPD),
FOR_ALL_ZONES(PGSTEAL_DIRECT),
diff --git a/include/uapi/asm-generic/mman-common.h b/include/uapi/asm-generic/mman-common.h
index ddc3b36f1046..7a94102b7a02 100644
--- a/include/uapi/asm-generic/mman-common.h
+++ b/include/uapi/asm-generic/mman-common.h
@@ -34,6 +34,7 @@
#define MADV_SEQUENTIAL 2 /* expect sequential page references */
#define MADV_WILLNEED 3 /* will need these pages */
#define MADV_DONTNEED 4 /* don't need these pages */
+#define MADV_FREE 5 /* free pages only if memory pressure */
/* common parameters: try to keep these consistent across architectures */
#define MADV_REMOVE 9 /* remove these pages & resources */
diff --git a/mm/madvise.c b/mm/madvise.c
index 0938b30da4ab..a21584235bb6 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -19,6 +19,14 @@
#include <linux/blkdev.h>
#include <linux/swap.h>
#include <linux/swapops.h>
+#include <linux/mmu_notifier.h>
+
+#include <asm/tlb.h>
+
+struct madvise_free_private {
+ struct vm_area_struct *vma;
+ struct mmu_gather *tlb;
+};
/*
* Any behaviour which results in changes to the vma->vm_flags needs to
@@ -31,6 +39,7 @@ static int madvise_need_mmap_write(int behavior)
case MADV_REMOVE:
case MADV_WILLNEED:
case MADV_DONTNEED:
+ case MADV_FREE:
return 0;
default:
/* be safe, default to 1. list exceptions explicitly */
@@ -251,6 +260,128 @@ static long madvise_willneed(struct vm_area_struct *vma,
return 0;
}
+static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
+ unsigned long end, struct mm_walk *walk)
+
+{
+ struct madvise_free_private *fp = walk->private;
+ struct mmu_gather *tlb = fp->tlb;
+ struct mm_struct *mm = tlb->mm;
+ struct vm_area_struct *vma = fp->vma;
+ spinlock_t *ptl;
+ pte_t *pte, ptent;
+ struct page *page;
+
+ split_huge_page_pmd(vma, addr, pmd);
+ if (pmd_trans_unstable(pmd))
+ return 0;
+
+ pte = pte_offset_map_lock(mm, pmd, addr, &ptl);
+ arch_enter_lazy_mmu_mode();
+ for (; addr != end; pte++, addr += PAGE_SIZE) {
+ ptent = *pte;
+
+ if (!pte_present(ptent))
+ continue;
+
+ page = vm_normal_page(vma, addr, ptent);
+ if (!page)
+ continue;
+
+ if (PageSwapCache(page)) {
+ if (!trylock_page(page))
+ continue;
+
+ if (!try_to_free_swap(page)) {
+ unlock_page(page);
+ continue;
+ }
+
+ ClearPageDirty(page);
+ unlock_page(page);
+ }
+
+ /*
+ * Some of architecture(ex, PPC) don't update TLB
+ * with set_pte_at and tlb_remove_tlb_entry so for
+ * the portability, remap the pte with old|clean
+ * after pte clearing.
+ */
+ ptent = ptep_get_and_clear_full(mm, addr, pte,
+ tlb->fullmm);
+ ptent = pte_mkold(ptent);
+ ptent = pte_mkclean(ptent);
+ set_pte_at(mm, addr, pte, ptent);
+ tlb_remove_tlb_entry(tlb, pte, addr);
+ }
+ arch_leave_lazy_mmu_mode();
+ pte_unmap_unlock(pte - 1, ptl);
+ cond_resched();
+ return 0;
+}
+
+static void madvise_free_page_range(struct mmu_gather *tlb,
+ struct vm_area_struct *vma,
+ unsigned long addr, unsigned long end)
+{
+ struct madvise_free_private fp = {
+ .vma = vma,
+ .tlb = tlb,
+ };
+
+ struct mm_walk free_walk = {
+ .pmd_entry = madvise_free_pte_range,
+ .mm = vma->vm_mm,
+ .private = &fp,
+ };
+
+ BUG_ON(addr >= end);
+ tlb_start_vma(tlb, vma);
+ walk_page_range(addr, end, &free_walk);
+ tlb_end_vma(tlb, vma);
+}
+
+static int madvise_free_single_vma(struct vm_area_struct *vma,
+ unsigned long start_addr, unsigned long end_addr)
+{
+ unsigned long start, end;
+ struct mm_struct *mm = vma->vm_mm;
+ struct mmu_gather tlb;
+
+ if (vma->vm_flags & (VM_LOCKED|VM_HUGETLB|VM_PFNMAP))
+ return -EINVAL;
+
+ /* MADV_FREE works for only anon vma at the moment */
+ if (vma->vm_file)
+ return -EINVAL;
+
+ start = max(vma->vm_start, start_addr);
+ if (start >= vma->vm_end)
+ return -EINVAL;
+ end = min(vma->vm_end, end_addr);
+ if (end <= vma->vm_start)
+ return -EINVAL;
+
+ lru_add_drain();
+ tlb_gather_mmu(&tlb, mm, start, end);
+ update_hiwater_rss(mm);
+
+ mmu_notifier_invalidate_range_start(mm, start, end);
+ madvise_free_page_range(&tlb, vma, start, end);
+ mmu_notifier_invalidate_range_end(mm, start, end);
+ tlb_finish_mmu(&tlb, start, end);
+
+ return 0;
+}
+
+static long madvise_free(struct vm_area_struct *vma,
+ struct vm_area_struct **prev,
+ unsigned long start, unsigned long end)
+{
+ *prev = vma;
+ return madvise_free_single_vma(vma, start, end);
+}
+
/*
* Application no longer needs these pages. If the pages are dirty,
* it's OK to just throw them away. The app will be more careful about
@@ -381,6 +512,14 @@ madvise_vma(struct vm_area_struct *vma, struct vm_area_struct **prev,
return madvise_remove(vma, prev, start, end);
case MADV_WILLNEED:
return madvise_willneed(vma, prev, start, end);
+ case MADV_FREE:
+ /*
+ * XXX: In this implementation, MADV_FREE works like
+ * MADV_DONTNEED on swapless system or full swap.
+ */
+ if (get_nr_swap_pages() > 0)
+ return madvise_free(vma, prev, start, end);
+ /* passthrough */
case MADV_DONTNEED:
return madvise_dontneed(vma, prev, start, end);
default:
@@ -400,6 +539,7 @@ madvise_behavior_valid(int behavior)
case MADV_REMOVE:
case MADV_WILLNEED:
case MADV_DONTNEED:
+ case MADV_FREE:
#ifdef CONFIG_KSM
case MADV_MERGEABLE:
case MADV_UNMERGEABLE:
diff --git a/mm/rmap.c b/mm/rmap.c
index 3e8491c504f8..04c181133890 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -663,6 +663,7 @@ int page_mapped_in_vma(struct page *page, struct vm_area_struct *vma)
}
struct page_referenced_arg {
+ int dirtied;
int mapcount;
int referenced;
unsigned long vm_flags;
@@ -677,6 +678,7 @@ static int page_referenced_one(struct page *page, struct vm_area_struct *vma,
struct mm_struct *mm = vma->vm_mm;
spinlock_t *ptl;
int referenced = 0;
+ int dirty = 0;
struct page_referenced_arg *pra = arg;
if (unlikely(PageTransHuge(page))) {
@@ -700,6 +702,11 @@ static int page_referenced_one(struct page *page, struct vm_area_struct *vma,
/* go ahead even if the pmd is pmd_trans_splitting() */
if (pmdp_clear_flush_young_notify(vma, address, pmd))
referenced++;
+
+ /*
+ * In this implmentation, MADV_FREE doesn't support THP free
+ */
+ dirty++;
spin_unlock(ptl);
} else {
pte_t *pte;
@@ -729,6 +736,10 @@ static int page_referenced_one(struct page *page, struct vm_area_struct *vma,
if (likely(!(vma->vm_flags & VM_SEQ_READ)))
referenced++;
}
+
+ if (pte_dirty(*pte))
+ dirty++;
+
pte_unmap_unlock(pte, ptl);
}
@@ -737,6 +748,9 @@ static int page_referenced_one(struct page *page, struct vm_area_struct *vma,
pra->vm_flags |= vma->vm_flags;
}
+ if (dirty)
+ pra->dirtied++;
+
pra->mapcount--;
if (!pra->mapcount)
return SWAP_SUCCESS; /* To break the loop */
@@ -761,6 +775,7 @@ static bool invalid_page_referenced_vma(struct vm_area_struct *vma, void *arg)
* @is_locked: caller holds lock on the page
* @memcg: target memory cgroup
* @vm_flags: collect encountered vma->vm_flags who actually referenced the page
+ * @is_pte_dirty: ptes which have marked dirty bit - used for lazyfree page
*
* Quick test_and_clear_referenced for all mappings to a page,
* returns the number of ptes which referenced the page.
@@ -768,7 +783,8 @@ static bool invalid_page_referenced_vma(struct vm_area_struct *vma, void *arg)
int page_referenced(struct page *page,
int is_locked,
struct mem_cgroup *memcg,
- unsigned long *vm_flags)
+ unsigned long *vm_flags,
+ int *is_pte_dirty)
{
int ret;
int we_locked = 0;
@@ -783,6 +799,9 @@ int page_referenced(struct page *page,
};
*vm_flags = 0;
+ if (is_pte_dirty)
+ *is_pte_dirty = 0;
+
if (!page_mapped(page))
return 0;
@@ -810,6 +829,9 @@ int page_referenced(struct page *page,
if (we_locked)
unlock_page(page);
+ if (is_pte_dirty)
+ *is_pte_dirty = pra.dirtied;
+
return pra.referenced;
}
@@ -1128,6 +1150,7 @@ static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
spinlock_t *ptl;
int ret = SWAP_AGAIN;
enum ttu_flags flags = (enum ttu_flags)arg;
+ int dirty = 0;
pte = page_check_address(page, mm, address, &ptl, 0);
if (!pte)
@@ -1157,7 +1180,8 @@ static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
pteval = ptep_clear_flush(vma, address, pte);
/* Move the dirty bit to the physical page now the pte is gone. */
- if (pte_dirty(pteval))
+ dirty = pte_dirty(pteval);
+ if (dirty)
set_page_dirty(page);
/* Update high watermark before we lower rss */
@@ -1186,6 +1210,19 @@ static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
swp_entry_t entry = { .val = page_private(page) };
pte_t swp_pte;
+ if (flags & TTU_FREE) {
+ VM_BUG_ON_PAGE(PageSwapCache(page), page);
+ if (!dirty && !PageDirty(page)) {
+ /* It's a freeable page by MADV_FREE */
+ dec_mm_counter(mm, MM_ANONPAGES);
+ goto discard;
+ } else {
+ set_pte_at(mm, address, pte, pteval);
+ ret = SWAP_FAIL;
+ goto out_unmap;
+ }
+ }
+
if (PageSwapCache(page)) {
/*
* Store the swap location in the pte.
@@ -1227,6 +1264,7 @@ static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
} else
dec_mm_counter(mm, MM_FILEPAGES);
+discard:
page_remove_rmap(page);
page_cache_release(page);
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 2836b5373b2e..76c2cc858323 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -713,13 +713,17 @@ enum page_references {
};
static enum page_references page_check_references(struct page *page,
- struct scan_control *sc)
+ struct scan_control *sc,
+ bool *freeable)
{
int referenced_ptes, referenced_page;
unsigned long vm_flags;
+ int pte_dirty;
+
+ VM_BUG_ON_PAGE(!PageLocked(page), page);
referenced_ptes = page_referenced(page, 1, sc->target_mem_cgroup,
- &vm_flags);
+ &vm_flags, &pte_dirty);
referenced_page = TestClearPageReferenced(page);
/*
@@ -760,6 +764,10 @@ static enum page_references page_check_references(struct page *page,
return PAGEREF_KEEP;
}
+ if (PageAnon(page) && !pte_dirty && !PageSwapCache(page) &&
+ !PageDirty(page))
+ *freeable = true;
+
/* Reclaim if clean, defer dirty pages to writeback */
if (referenced_page && !PageSwapBacked(page))
return PAGEREF_RECLAIM_CLEAN;
@@ -828,6 +836,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
int may_enter_fs;
enum page_references references = PAGEREF_RECLAIM_CLEAN;
bool dirty, writeback;
+ bool freeable = false;
cond_resched();
@@ -950,7 +959,8 @@ static unsigned long shrink_page_list(struct list_head *page_list,
}
if (!force_reclaim)
- references = page_check_references(page, sc);
+ references = page_check_references(page, sc,
+ &freeable);
switch (references) {
case PAGEREF_ACTIVATE:
@@ -966,7 +976,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
* Anonymous process memory has backing store?
* Try to allocate it some swap space here.
*/
- if (PageAnon(page) && !PageSwapCache(page)) {
+ if (PageAnon(page) && !PageSwapCache(page) && !freeable) {
if (!(sc->gfp_mask & __GFP_IO))
goto keep_locked;
if (!add_to_swap(page, page_list))
@@ -981,8 +991,9 @@ static unsigned long shrink_page_list(struct list_head *page_list,
* The page is mapped into the page tables of one or more
* processes. Try to unmap it here.
*/
- if (page_mapped(page) && mapping) {
- switch (try_to_unmap(page, ttu_flags)) {
+ if (page_mapped(page) && (mapping || freeable)) {
+ switch (try_to_unmap(page,
+ freeable ? TTU_FREE : ttu_flags)) {
case SWAP_FAIL:
goto activate_locked;
case SWAP_AGAIN:
@@ -990,7 +1001,20 @@ static unsigned long shrink_page_list(struct list_head *page_list,
case SWAP_MLOCK:
goto cull_mlocked;
case SWAP_SUCCESS:
- ; /* try to free the page below */
+ /* try to free the page below */
+ if (!freeable)
+ break;
+ /*
+ * Freeable anon page doesn't have mapping
+ * due to skipping of swapcache so we free
+ * page in here rather than __remove_mapping.
+ */
+ VM_BUG_ON_PAGE(PageSwapCache(page), page);
+ if (!page_freeze_refs(page, 1))
+ goto keep_locked;
+ __clear_page_locked(page);
+ count_vm_event(PGLAZYFREED);
+ goto free_it;
}
}
@@ -1730,7 +1754,7 @@ static void shrink_active_list(unsigned long nr_to_scan,
}
if (page_referenced(page, 0, sc->target_mem_cgroup,
- &vm_flags)) {
+ &vm_flags, NULL)) {
nr_rotated += hpage_nr_pages(page);
/*
* Identify referenced, file-backed active pages and
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 8d82c91344f1..eb295a844a38 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -811,6 +811,7 @@ const char * const vmstat_text[] = {
"pgfault",
"pgmajfault",
+ "pglazyfreed",
TEXTS_FOR_ZONES("pgrefill")
TEXTS_FOR_ZONES("pgsteal_kswapd")
--
2.0.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v15 0/7] MADV_FREE support
From: Minchan Kim @ 2014-08-18 0:17 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Michael Kerrisk,
linux-api-u79uwXL29TY76Z2rM5mHXA, Hugh Dickins, Johannes Weiner,
Rik van Riel, KOSAKI Motohiro, Mel Gorman, Jason Evans,
zhangyanfei-BthXqXjhjHXQFUHtdCDX3A, Kirill A. Shutemov,
Minchan Kim
This patch enable MADV_FREE hint for madvise syscall, which have
been supported by other OSes. [PATCH 1] includes the details.
[1] support MADVISE_FREE for !THP page so if VM encounter
THP page in syscall context, it splits THP page.
[2-6] is to preparing to call madvise syscall without THP plitting
[7] enable THP page support for MADV_FREE.
* from v14
* Add more Ackedy-by from arch people(sparc, arm64 and arm)
* Drop s390 since pmd_dirty/clean was merged
* from v13
* Add more Ackedy-by from arch people(arm, arm64 and ppc)
* Rebased on mmotm 2014-08-13-14-29
* from v12
* Fix - skip to mark free pte on try_to_free_swap failed page - Kirill
* Add more Acked-by from arch maintainers and Kirill
* From v11
* Fix arm build - Steve
* Separate patch for arm and arm64 - Steve
* Remove unnecessary check - Kirill
* Skip non-vm_normal page - Kirill
* Add Acked-by - Zhang
* Sparc64 build fix
* Pagetable walker THP handling fix
* From v10
* Add Acked-by from arch stuff(x86, s390)
* Pagewalker based pagetable working - Kirill
* Fix try_to_unmap_one broken with hwpoison - Kirill
* Use VM_BUG_ON_PAGE in madvise_free_pmd - Kirill
* Fix pgtable-3level.h for arm - Steve
* From v9
* Add Acked-by - Rik
* Add THP page support - Kirill
* From v8
* Rebased-on v3.16-rc2-mmotm-2014-06-25-16-44
* From v7
* Rebased-on next-20140613
* From v6
* Remove page from swapcache in syscal time
* Move utility functions from memory.c to madvise.c - Johannes
* Rename untilify functtions - Johannes
* Remove unnecessary checks from vmscan.c - Johannes
* Rebased-on v3.15-rc5-mmotm-2014-05-16-16-56
* Drop Reviewe-by because there was some changes since then.
* From v5
* Fix PPC problem which don't flush TLB - Rik
* Remove unnecessary lazyfree_range stub function - Rik
* Rebased on v3.15-rc5
* From v4
* Add Reviewed-by: Zhang Yanfei
* Rebase on v3.15-rc1-mmotm-2014-04-15-16-14
* From v3
* Add "how to work part" in description - Zhang
* Add page_discardable utility function - Zhang
* Clean up
* From v2
* Remove forceful dirty marking of swap-readed page - Johannes
* Remove deactivation logic of lazyfreed page
* Rebased on 3.14
* Remove RFC tag
* From v1
* Use custom page table walker for madvise_free - Johannes
* Remove PG_lazypage flag - Johannes
* Do madvise_dontneed instead of madvise_freein swapless system
Minchan Kim (7):
mm: support madvise(MADV_FREE)
x86: add pmd_[dirty|mkclean] for THP
sparc: add pmd_[dirty|mkclean] for THP
powerpc: add pmd_[dirty|mkclean] for THP
arm: add pmd_mkclean for THP
arm64: add pmd_[dirty|mkclean] for THP
mm: Don't split THP page when syscall is called
arch/arm/include/asm/pgtable-3level.h | 1 +
arch/arm64/include/asm/pgtable.h | 2 +
arch/powerpc/include/asm/pgtable-ppc64.h | 2 +
arch/sparc/include/asm/pgtable_64.h | 16 ++++
arch/x86/include/asm/pgtable.h | 10 ++
include/linux/huge_mm.h | 4 +
include/linux/rmap.h | 9 +-
include/linux/vm_event_item.h | 1 +
include/uapi/asm-generic/mman-common.h | 1 +
mm/huge_memory.c | 35 +++++++
mm/madvise.c | 159 +++++++++++++++++++++++++++++++
mm/rmap.c | 46 ++++++++-
mm/vmscan.c | 64 +++++++++----
mm/vmstat.c | 1 +
14 files changed, 331 insertions(+), 20 deletions(-)
--
2.0.0
^ permalink raw reply
* Re: [PATCH v14 5/8] s390: add pmd_[dirty|mkclean] for THP
From: Minchan Kim @ 2014-08-18 0:06 UTC (permalink / raw)
To: Martin Schwidefsky
Cc: Andrew Morton, linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Michael Kerrisk, Linux API,
Hugh Dickins, Johannes Weiner, Rik van Riel, KOSAKI Motohiro,
Mel Gorman, Jason Evans, Zhang Yanfei, Kirill A. Shutemov,
Heiko Carstens, Dominik Dingel, Christian Borntraeger,
linux-s390-u79uwXL29TY76Z2rM5mHXA, Gerald Schaefer
In-Reply-To: <20140814091614.4a0d5178@mschwide>
Hello,
On Thu, Aug 14, 2014 at 09:16:14AM +0200, Martin Schwidefsky wrote:
> On Thu, 14 Aug 2014 10:53:29 +0900
> Minchan Kim <minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
>
> > MADV_FREE needs pmd_dirty and pmd_mkclean for detecting recent
> > overwrite of the contents since MADV_FREE syscall is called for
> > THP page but for s390 pmds only referenced bit is available
> > because there is no free bit left in the pmd entry for the
> > software dirty bit so this patch adds dumb pmd_dirty which
> > returns always true by suggesting by Martin.
> >
> > They finally find a solution in future.
> > http://marc.info/?l=linux-api&m=140440328820808&w=2
>
> The solution is already there, see git commit 152125b7a882df36.
> You can drop this patch.
Thanks for the heads up. I will drop it in next spin.
>
> --
> blue skies,
> Martin.
>
> "Reality continues to ruin my life." - Calvin.
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo-Bw31MaZKKs0EbZ0PF+XxCw@public.gmane.org For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org"> email-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org </a>
--
Kind regards,
Minchan Kim
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox