container disk quota

container disk quota

[jeff.liu-QHcLZuEGTsvQT0dZR+AlfA]

Hello All,

According to glauber's comments regarding container disk quota, it should be binded to mount
namespace rather than cgroup.

Per my try out, it works just fine by combining with userland quota utilitly in this way.
However, they are something has to be done at user tools too IMHO.

Currently, the patchset is in very initial phase, I'd like to post it early to seek more
feedbacks from you guys.

Hopefully I can clarify my ideas clearly.

Kernel part:
* Container quota can be enabled indenpent to VFS quota or particular file system quota.
  quota per user/group are kept at memory instead of saved at separately files like general quota.
  There is no need to remount the rootfs inside container with general quota strings, quota could be
  enabled through quotaon/off directly.

* Always honor underlying file system quota checking firstly.  i.e, the exported quota bill up
  routines are take affected only after file system quota check up done if it is enabled at the
  same time.   hence the space allocation or inode creation inside container will failed if the
  outside quota limits were exceeded.

* Make use of the general VFS Q_XXXX quota control flags.

* Introduce a new disk quota struture as well as the operations to mount namespacedata structure,
  it should only be allocated and initialized at CLONE stage for contianer.

* Modify quotactl(2) to examine if the caller is invoked inside container.
  implemented by checking the quota device name("rootfs" for lxc guest) or current pid namespace
  is not the initial one, then do mount namespace quotactl if required, or goto
  the normal quotactl procedure.

* Introduce a new quota format "QFMT_NS" for container.  It will be used to examine the quota
  format at userland tools, so that quotacheck will do container quota IO initialization and
  proceeding operations. This flag returned when Q_GETQINFO was issued.

* Export a couple of container quota bill routines to the desired underlying
  file system.  They will take affected if container quota is enabled at kernel
  configuration, or just some inline functions without much overhead.

* Also, I have not handle a couple of things for now.
  . I think the container quota should be isolated to Jan's fs/quota/ directory.
  . There are a dozens of helper routines at general quota, e.g,
    struct if_dqblk <-> struct fs_disk_quota converts.
    dquot space and inodes bill up.
    They can be refactored as shared routines to some extents.
  . quotastats(8) is not teached to aware container for now.

Changes in quota userland utility:
* Introduce a new quota format string "lxc" to all quota control utility, to
  let each utility know that the user want to run container quota control. e.g:
  quotacheck -cvugm -F "lxc" /
  quotaon -u -F "lxc" /
  ....

* Currently, I manually created the underlying device(by editing cgroup
  device access list and running mknod /dev/sdaX x x) for the rootfs
  inside containers to let the cache mount points routine pass for
  executing quotacheck against the "/" directory.  Actually, it can be
  omitted here.

* Add a new quotaio_lxc.c[.h] for container quota IO, it basically same to
  VFS quotaio logic, I just hope to isolate container stuff here.

Issues:
* How to detect quotactl(2) is launched from container in a reasonable way.

* Do we need to let container quota works for cgroup combine with unshare(1)?
  Now the patchset is mainly works for lxc guest.  IMHO, it can be used outside
  guest if the user desired.  In this case, the quota limits can take effort
  among different underlying file systems if they have exported quota billing
  routines.

* As the configure entry for print warnning info to TTY has been marked to
  obsoleted, do we still need to support that.

* The warnning info format for sending it through netlink interface.
  VFS quota has a device parameter filled in the warns, how we define the
  format for container?

* The hash table list defines(hash table size)for dquot caching for each type is
  referred to kernel/user.c, maybe its better to define an array separatly for
  performance optimizations.  Of course, that's all depending on my current
  implementation is on the right road. :)
 
* Container quota statistics, should them be calculated and exposed to /proc/fs/quota?  If the underlying file system also enabled with quotas, they will be
  mixed up, so how about add a new proc file like "ns_quota" there?

* Memory shrinks acquired from kswap.
  As all dquot are cached in memory, and if the user executing quotaoff, maybe
  I need to handle quota disable but still be kept at memory.
  Also, add another routine to disable and remove all quotas from memory to
  save memory directly.

* Project quota(i.e, tree quota) support.
  Now the quota implemented without project quota supports, but it can be
  supported not complex based on current code, add a new parameter to
  ns_dquot_alloc_block(), etc... is ok.
  However, XFS support project quota setup on xfs tools, I observed there
  already have patchset for this feature in EXT4 mailist, is it possble
  to supply a unique interface and implementation to quota tools in the
  furture?
  AFAICS, project quota can be setup in container, because of we can
  fetch the super block from the transferred path.  Hence, the desired
  ioctl(2) for underlying file system can be invoked. 

* Security check up for mount namespace quotactl(2).
  In this version, I only do basic security check up to see if the caller
  has properly permissions for doing that.  I think I must miss much things
  in this point.

Testing:
Currently patch is lacking tests, I only do a few check to make sure the
basic operations works.

First of all, we need to invoke quotacheck with "--no-remount" opition
since the rootfs inside container guest can not be remouted:
root@debian:~/# quotacheck -cvugm -F "lxc" /
quotacheck: quotacheck: Scanning rootfs [/] done
quotacheck: Old user file name could not been determined. Usage will not be subtracted.
quotacheck: Old group file name could not been determined. Usage will not be subtracted.
quotacheck: Old user file name could not been determined. Usage will not be subtracted.
quotacheck: Old group file name could not been determined. Usage will not be subtracted.
quotacheck: Checked 3370 directories and 39434 files

By default, user/group quota is off:
root@debian:~/# quotaon -u -F "lxc" -p /
user quota on / (rootfs) is off

root@debian:~/# quotaon -u -F "lxc" -p /
group quota on / (rootfs) is off

Turn them on: 
root@debian:~/# quotaon -u -F "lxc" /   
root@debian:~/# quotaon -g -F "lxc" /
root@debian:~/# quotaon -u -F "lxc" -p /
user quota on / (rootfs) is on
root@debian:~/# quotaon -g -F "lxc" -p /
group quota on / (rootfs) is on

Edit quota, soft/hard for both space and inode are zeros by default:
configure them to a desired value:
root@debian:~/# edquota -u -F "lxc" /
Disk quotas for user jeff (uid 1000):
  Filesystem                   blocks       soft       hard     inodes     soft  
    hard
  rootfs                   2025740    2025840   2026000  42786  42790   42800

The configuration are saved properly:
root@debian:~/# repquota -u -F "lxc" /
Block grace time: 00:00; Inode grace time: 00:00
                        Block limits                File limits
User            used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --      44       0       0             20     0     0       
jeff      -- 2025740 2025840 2026000          42786 42790 42800  

Do checking for blocks and inodes limits:
root@debian:~/# su - jeff
jeff@debian:/$ dd if=/dev/zero of=abc bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 1.19014 s, 8.8 MB/s
root@debian:~/# repquota -u -F "lxc" /
Jeff *** report() type=0 handle index=0
*** Report for user quotas on device rootfs
Block grace time: 00:00; Inode grace time: 00:00
                        Block limits                File limits
User            used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --      44       0       0             20     0     0       
jeff      +- 2025980 2025840 2026000  7days   42786 42790 42800       

root@debian:~/# repquota -g -F "lxc" /
*** Report for group quotas on device rootfs
Block grace time: 00:00; Inode grace time: 00:00
                        Block limits                File limits
Group           used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --    8564       0       0            390     0     0       
adm       --     220       0       0              6     0     0       
tty       --       0       0       0              1     0     0       
utmp      --       4       0       0              1     0     0       
jeff      -- 2021268       0       0          42716     0     0  

root@debian:~/# su - jeff
jeff@debian:/$ dd if=/dev/zero of=test_space bs=1M count=100
dd: writing `test_space': Disk quota exceeded
11+0 records in
10+0 records out
10506240 bytes (11 MB) copied, 1.24721 s, 8.4 MB/s

root@debian:~/# repquota -u -F "lxc" /
Jeff *** report() type=0 handle index=0
*** Report for user quotas on device rootfs
Block grace time: 00:00; Inode grace time: 00:00
                        Block limits                File limits
User            used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --      44       0       0             20     0     0       
jeff      +- 2026000 2025840 2026000  7days   42786 42790 42800   

root@debian:~/# su - jeff
jeff@debian:/$ for ((i=0; i<20; i++)); do touch test_file_cnt.$i; done
touch: cannot touch `test_file_cnt.14': Disk quota exceeded
touch: cannot touch `test_file_cnt.16': Disk quota exceeded
touch: cannot touch `test_file_cnt.18': Disk quota exceeded

root@debian:~/# repquota -u -F "lxc" /
Block grace time: 00:00; Inode grace time: 00:00
                        Block limits                File limits
User            used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --      44       0       0             20     0     0       
jeff      ++ 2026000 2025840 2026000  6days   42800 42790 42800  7days

Any comments are appreciated, have a nice day!

-Jeff

[PATCH 01/12] container quota: add kernel configuration for container quota.

[jeff.liu-QHcLZuEGTsvQT0dZR+AlfA]

Add kernel configuration for container quota.

Signed-off-by: Jie Liu <jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 fs/quota/Kconfig |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/fs/quota/Kconfig b/fs/quota/Kconfig
index 880fd98..2dd3e83 100644
--- a/fs/quota/Kconfig
+++ b/fs/quota/Kconfig
@@ -16,6 +16,15 @@ config QUOTA
 	  with the quota tools. Probably the quota support is only useful for
 	  multi user systems. If unsure, say N.
 
+config NS_QUOTA
+	bool "Mount namespace quota support"
+	depends on QUOTA
+	default n
+	help
+	  If you say Y here, you will be able to set per container user limits
+	  for disk usage (also called disk quotas). Currently, it works for the
+	  ext4 file system only for demonstration purpose.
+
 config QUOTA_NETLINK_INTERFACE
 	bool "Report quota messages through netlink interface"
 	depends on QUOTACTL && NET
-- 
1.7.9

[PATCH 02/12] container quota: lock/unlock mount namespace when performing quotactl.

[jeff.liu-QHcLZuEGTsvQT0dZR+AlfA]

Introduce two functions to lock/unlock a particular mount namespace
when container quota operations is performed.

Signed-off-by: Jie Liu <jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 fs/namespace.c                |   17 +++++++++++++++++
 include/linux/mnt_namespace.h |   13 +++++++++++++
 2 files changed, 30 insertions(+), 0 deletions(-)

diff --git a/fs/namespace.c b/fs/namespace.c
index e608199..8823ba8 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -2633,3 +2633,20 @@ bool our_mnt(struct vfsmount *mnt)
 {
 	return check_mnt(real_mount(mnt));
 }
+
+/*
+ * Lock the mount namespace when performing a particular container
+ * disk quotactl operation, especially to protect removing it.
+ */
+void lock_mnt_ns(struct mnt_namespace *ns)
+{
+	down_write(&namespace_sem);
+}
+
+/*
+ * Unlock the mount namespace after quotactl on that was done.
+ */
+void unlock_mnt_ns(struct mnt_namespace *ns)
+{
+	up_write(&namespace_sem);
+}
diff --git a/include/linux/mnt_namespace.h b/include/linux/mnt_namespace.h
index 5a8e390..ff72ed5 100644
--- a/include/linux/mnt_namespace.h
+++ b/include/linux/mnt_namespace.h
@@ -9,6 +9,19 @@ extern struct mnt_namespace *copy_mnt_ns(unsigned long, struct mnt_namespace *,
 		struct fs_struct *);
 extern void put_mnt_ns(struct mnt_namespace *ns);
 
+#ifdef CONFIG_NS_QUOTA
+extern void lock_mnt_ns(struct mnt_namespace *);
+extern void unlock_mnt_ns(struct mnt_namespace *);
+#else
+void lock_mnt_ns(struct mnt_namespace *)
+{
+}
+
+void unlock_mnt_ns(struct mnt_namespace *)
+{
+}
+#endif
+
 extern const struct file_operations proc_mounts_operations;
 extern const struct file_operations proc_mountinfo_operations;
 extern const struct file_operations proc_mountstats_operations;
-- 
1.7.9

[PATCH 03/12] container quota: introduce container quota format identifier.

[jeff.liu-QHcLZuEGTsvQT0dZR+AlfA]

Introduce a new quota format QFMT_NS, return it for Q_GETINFO.
Further more, linux quota tools need such info to decide which
kind of quota control operations will be initialized and performed.

Signed-off-by: Jie Liu <jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 include/linux/quota.h |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/include/linux/quota.h b/include/linux/quota.h
index c09fa04..84e9be7 100644
--- a/include/linux/quota.h
+++ b/include/linux/quota.h
@@ -75,6 +75,7 @@
 #define	QFMT_VFS_V0 2
 #define QFMT_OCFS2 3
 #define	QFMT_VFS_V1 4
+#define QFMT_NS	5
 
 /* Size of block in which space limits are passed through the quota
  * interface */
-- 
1.7.9

[PATCH 04/12] container quota: introduce container disk quota data header file.

[jeff.liu-QHcLZuEGTsvQT0dZR+AlfA]

Header file for container disk quota.

Signed-off-by: Jie Liu <jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 fs/ns_quota.h |   84 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 84 insertions(+), 0 deletions(-)
 create mode 100644 fs/ns_quota.h

diff --git a/fs/ns_quota.h b/fs/ns_quota.h
new file mode 100644
index 0000000..cf90fff
--- /dev/null
+++ b/fs/ns_quota.h
@@ -0,0 +1,84 @@
+#ifndef _LINUX_NS_QUOTA_
+#define _LINUX_NS_QUOTA_
+
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/rwsem.h>
+#include <linux/spinlock.h>
+#include <linux/atomic.h>
+#include <linux/quota.h>
+#include <linux/user_namespace.h>
+
+#define NS_MAX_IQ_TIME	604800		/* (7*24*60*60) 1 week */
+#define NS_MAX_DQ_TIME	604800		/* (7*24*60*60) 1 week */
+
+/*
+ * Below most stuff were copied from kernel/user.c, using them
+ * to calculate and define quota hash table and hash routine.
+ */
+#define NS_DQHASH_BITS	(CONFIG_BASE_SMALL ? 3 : 7)
+#define NS_DQHASH_SZ	(1 << NS_DQHASH_BITS)
+
+struct ns_mem_dqinfo {
+	unsigned long dqi_flags;
+	unsigned int dqi_bgrace;
+	unsigned int dqi_igrace;
+	qsize_t dqi_maxblimit;
+	qsize_t dqi_maxilimit;
+};
+
+/* Mount namespace disk quota definitation */
+struct ns_quota_info {
+	/* Flags for disk quotas on this mount namespace */
+	unsigned int dq_flags;
+	/* Serialize mount namespace quotaon & quotaoff */
+	struct mutex dqonoff_mutex;
+	/* Serialize user/group quota list operations */
+	spinlock_t dq_list_lock;
+	/* Serialize quota value incr/decr operations */
+	spinlock_t dq_data_lock;
+	struct kmem_cache *dquot_cachep;
+	/* Hash list of all user disk quotas */
+	struct hlist_head u_dquots[NS_DQHASH_SZ];
+	/* Hash list of all group disk quotas */
+	struct hlist_head g_dquots[NS_DQHASH_SZ];
+	/* Information for each quota type */
+	struct ns_mem_dqinfo dqinfo[MAXQUOTAS];
+};
+
+/* Mount namespace disk quota object */
+struct ns_dquot {
+	struct hlist_node dq_hash_node;	/* Hash list in memory */
+	struct mnt_namespace *dq_ns;	/* Mount namespace this applies to */
+	unsigned int dq_id;		/* ID this applies to (uid, gid) */
+	unsigned long dq_flags;		/* See DQ_* */
+	short dq_type;			/* Type of quota */
+	struct mem_dqblk dq_dqb;	/* Diskquota usage */
+};
+
+/* Operations for quota object allocation/destroy */
+struct ns_dquot_ops {
+	/* Allocate memory for new dqout object */
+	struct ns_dquot *(*alloc_dquot)(struct mnt_namespace *);
+	/* Free memory for dquot */
+	void (*destroy_dquot)(struct ns_dquot *);
+};
+
+/* Operations for quota control */
+struct ns_quotactl_ops {
+	int (*quota_on)(struct mnt_namespace *, int);
+	int (*quota_off)(struct mnt_namespace *, int);
+	int (*get_info)(struct mnt_namespace *, int, struct if_dqinfo *);
+	int (*set_info)(struct mnt_namespace *, int, struct if_dqinfo *);
+	int (*get_dqblk)(struct mnt_namespace *, int, qid_t,
+			 struct fs_disk_quota *);
+	int (*set_dqblk)(struct mnt_namespace *, int, qid_t,
+			 struct fs_disk_quota *);
+};
+
+int ns_dqinfo_init(struct mnt_namespace *ns);
+void ns_dqinfo_destroy(struct mnt_namespace *ns);
+extern unsigned int ns_dquot_getfmt(struct mnt_namespace *ns, int type);
+
+#endif /* _LINUX_NS_QUOTA_ */
-- 
1.7.9

[PATCH 05/12] container quota: bind disk quota stuff on mount namespace.

[jeff.liu-QHcLZuEGTsvQT0dZR+AlfA]

Bind disk quota stuff on mount namespace definition.

Signed-off-by: Jie Liu <jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 fs/mount.h |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/fs/mount.h b/fs/mount.h
index 4ef36d9..5b9de40 100644
--- a/fs/mount.h
+++ b/fs/mount.h
@@ -2,12 +2,24 @@
 #include <linux/seq_file.h>
 #include <linux/poll.h>
 
+#ifdef CONFIG_NS_QUOTA
+#include "ns_quota.h"
+#endif
+
 struct mnt_namespace {
 	atomic_t		count;
 	struct mount *	root;
 	struct list_head	list;
 	wait_queue_head_t poll;
 	int event;
+#ifdef CONFIG_NS_QUOTA
+	/* All things regarding disk quota on mount namespace */
+	struct ns_quota_info *ns_dqinfo;
+	/* Mount namespace disk quota control operations */
+	const struct ns_quotactl_ops *ns_qcop;
+	/* Mount namespace disk quota operations */
+	const struct ns_dquot_ops *ns_dqop;
+#endif
 };
 
 struct mnt_pcp {
-- 
1.7.9

[PATCH 06/12] container quota: implementations and header for block/inode bill up.

[jeff.liu-QHcLZuEGTsvQT0dZR+AlfA]

Add container disk quota operation header file as well as the implementations.

Signed-off-by: Jie Liu <jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 fs/ns_dquot.c    | 1246 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/ns_quotaops.h |   72 ++++
 2 files changed, 1318 insertions(+), 0 deletions(-)
 create mode 100644 fs/ns_dquot.c
 create mode 100644 fs/ns_quotaops.h

diff --git a/fs/ns_dquot.c b/fs/ns_dquot.c
new file mode 100644
index 0000000..27c36c6
--- /dev/null
+++ b/fs/ns_dquot.c
@@ -0,0 +1,1246 @@
+#include <linux/errno.h>
+#include <linux/kernel.h>
+#include <linux/fs.h>
+#include <linux/mount.h>
+#include <linux/mm.h>
+#include <linux/time.h>
+#include <linux/types.h>
+#include <linux/string.h>
+#include <linux/fcntl.h>
+#include <linux/stat.h>
+#include <linux/tty.h>
+#include <linux/file.h>
+#include <linux/slab.h>
+#include <linux/sysctl.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/proc_fs.h>
+#include <linux/security.h>
+#include <linux/sched.h>
+#include <linux/kmod.h>
+#include <linux/namei.h>
+#include <linux/capability.h>
+#include <linux/quota.h>
+#include <linux/quotaops.h>
+#include <linux/nsproxy.h>
+#include <linux/user_namespace.h>
+#include <linux/mnt_namespace.h>
+
+#include "mount.h"
+#include "internal.h" /* ugh */
+
+#include <linux/uaccess.h>
+
+static __cacheline_aligned_in_smp DEFINE_SPINLOCK(ns_dq_state_lock);
+
+#define VFS_FS_DQ_MASK \
+	(FS_DQ_BCOUNT | FS_DQ_BSOFT | FS_DQ_BHARD | \
+	 FS_DQ_ICOUNT | FS_DQ_ISOFT | FS_DQ_IHARD | \
+	 FS_DQ_BTIMER | FS_DQ_ITIMER)
+
+#define NS_DQHASH_MASK	(NS_DQHASH_BITS - 1)
+#define __hashfn(id)	(((id >> NS_DQHASH_BITS) + id) & NS_DQHASH_MASK)
+#define hashentry(dq_hash_table, id)	(dq_hash_table + __hashfn((id)))
+
+static inline void remove_ns_dquot_hash(struct ns_dquot *dquot)
+{
+	hlist_del_init(&dquot->dq_hash_node);
+}
+
+static struct ns_dquot *ns_dqhash_find(unsigned int id,
+				       struct hlist_head *hashent)
+{
+	struct ns_dquot *dquot;
+	struct hlist_node *h;
+
+	hlist_for_each_entry(dquot, h, hashent, dq_hash_node) {
+		/* FIXME: maybe need to add ns check up as well */
+		if (dquot->dq_id == id)
+			return dquot;
+	}
+
+	return NULL;
+}
+
+/*
+ * Find out a desired dquot.  Currently, it only supports user quota
+ * type, maybe we also need to add directory quota support here.
+ */
+static struct ns_dquot *find_ns_dquot(struct mnt_namespace *ns,
+				      unsigned int id, int type)
+{
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+	struct ns_dquot *dquot;
+
+	switch (type) {
+	case USRQUOTA:
+		dquot = ns_dqhash_find(id, hashentry(dqinfo->u_dquots, id));
+		break;
+	case GRPQUOTA:
+		dquot = ns_dqhash_find(id, hashentry(dqinfo->g_dquots, id));
+		break;
+	}
+
+	return dquot;
+}
+
+static void insert_ns_dquot_hash(struct ns_dquot *dquot)
+{
+	struct ns_quota_info *dqinfo = dquot->dq_ns->ns_dqinfo;
+	struct hlist_head *hashent;
+
+	switch (dquot->dq_type) {
+	case USRQUOTA:
+		hashent = hashentry(dqinfo->u_dquots, dquot->dq_id);
+		break;
+	case GRPQUOTA:
+		hashent = hashentry(dqinfo->g_dquots, dquot->dq_id);
+		break;
+	}
+
+	hlist_add_head(&dquot->dq_hash_node, hashent);
+}
+
+/* Allocate and return a new dquot */
+static inline struct ns_dquot *ns_dquot_alloc(struct mnt_namespace *ns)
+{
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+
+	return kmem_cache_zalloc(dqinfo->dquot_cachep, GFP_NOFS);
+}
+
+/* Remove a dquot from cache */
+static void ns_dquot_destroy(struct ns_dquot *dquot)
+{
+	struct ns_quota_info *dqinfo = dquot->dq_ns->ns_dqinfo;
+
+	if (dqinfo->dquot_cachep)
+		kmem_cache_free(dqinfo->dquot_cachep, dquot);
+}
+
+static void __remove_dq_hash_list_items(struct hlist_head *hashent)
+{
+	struct ns_dquot *dquot;
+	struct hlist_node *h, *tmp;
+
+	hlist_for_each_entry_safe(dquot, h, tmp, hashent, dq_hash_node)
+		remove_ns_dquot_hash(dquot);
+}
+
+static void __remove_dq_hash_list(struct hlist_head *hashent)
+{
+	if (!hlist_empty(hashent))
+		__remove_dq_hash_list_items(hashent);
+}
+
+static inline bool ns_has_quota_usage_enabled(struct mnt_namespace *ns,
+					      int type)
+{
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+
+	return dqinfo->dq_flags &
+	       dquot_state_flag(DQUOT_USAGE_ENABLED, type);
+}
+
+static inline bool ns_has_quota_limit_enabled(struct mnt_namespace *ns,
+					      int type)
+{
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+
+	return dqinfo->dq_flags &
+	       dquot_state_flag(DQUOT_LIMITS_ENABLED, type);
+}
+
+/*
+ * Does kernel know about any quota information for the given
+ * mount namespace + type?
+ */
+static inline bool ns_has_quota_loaded(struct mnt_namespace *ns, int type)
+{
+	/* currently if anything is on, then quota usage is on as well */
+	return ns_has_quota_usage_enabled(ns, type);
+}
+
+static inline unsigned ns_any_quota_loaded(struct mnt_namespace *ns)
+{
+	unsigned type, tmsk = 0;
+	for (type = 0; type < MAXQUOTAS; type++)
+		tmsk |= ns_has_quota_loaded(ns, type) << type;
+
+	return tmsk;
+}
+
+static inline bool ns_has_quota_active(struct mnt_namespace *ns, int type)
+{
+	return ns_has_quota_limit_enabled(ns, type);
+}
+
+/*
+ * FIXME: Currently, below warning stuff for mount namespace quota are not well
+ * configured and tested, the only purpose here is to demo the how we can using
+ * them in the furture.
+ */
+struct ns_dquot_warn {
+	struct mnt_namespace *w_ns;
+	qid_t w_dq_id;
+	short w_dq_type;
+	short w_type;
+};
+
+static int warning_issued(struct ns_dquot *dquot, const int warntype)
+{
+	int flag = (warntype == QUOTA_NL_BHARDWARN ||
+		    warntype == QUOTA_NL_BSOFTLONGWARN) ? DQ_BLKS_B :
+		    ((warntype == QUOTA_NL_IHARDWARN ||
+		      warntype == QUOTA_NL_ISOFTLONGWARN) ? DQ_INODES_B : 0);
+
+	if (!flag)
+		return 0;
+
+	return test_and_set_bit(flag, &dquot->dq_flags);
+}
+
+/* FIXME: below parameter is not presented on Kconfig yet. */
+#ifdef CONFIG_PRINT_NS_QUOTA_WARNING
+static int flag_print_warnings = 1;
+
+static int need_print_warning(struct dquot_warn *warn)
+{
+	if (!flag_print_warnings)
+		return 0;
+
+	switch (warn->w_dq_type) {
+	case USRQUOTA:
+		return current_fsuid() == warn->w_dq_id;
+	case GRPQUOTA:
+		return in_group_p(warn->w_dq_id);
+	}
+
+	return 0;
+}
+
+/*
+ * Print warning to user which exceeded quota.
+ * FIXME:
+ * As "Pint quota warning to console" has been marked to OBSOLETE on
+ * Kconfig menu, maybe we can just ignore that in mount namespace quota?
+ */
+static void print_warning(struct dquot_warn *warn)
+{
+	char *msg = NULL;
+	struct tty_struct *tty;
+	int warntype = warn->w_type;
+
+	if (warntype == QUOTA_NL_IHARDBELOW ||
+	    warntype == QUOTA_NL_ISOFTBELOW ||
+	    warntype == QUOTA_NL_BHARDBELOW ||
+	    warntype == QUOTA_NL_BSOFTBELOW ||
+	    !need_print_warning(warn))
+		return;
+
+	tty = get_current_tty();
+	if (!tty)
+		return;
+
+	tty_write_message(tty, warn->w_sb->s_id);
+	if (warntype == QUOTA_NL_ISOFTWARN || warntype == QUOTA_NL_BSOFTWARN)
+		tty_write_message(tty, ": warning, ");
+	else
+		tty_write_message(tty, ": write failed, ");
+
+	tty_write_message(tty, quotatypes[warn->w_dq_type]);
+	switch (warntype) {
+	case QUOTA_NL_IHARDWARN:
+		msg = " file limit reached.\r\n";
+		break;
+	case QUOTA_NL_ISOFTLONGWARN:
+		msg = " file quota exceeded too long.\r\n";
+		break;
+	case QUOTA_NL_ISOFTWARN:
+		msg = " file quota exceeded.\r\n";
+		break;
+	case QUOTA_NL_BHARDWARN:
+		msg = " block limit reached.\r\n";
+		break;
+	case QUOTA_NL_BSOFTLONGWARN:
+		msg = " block quota exceeded too long.\r\n";
+		break;
+	case QUOTA_NL_BSOFTWARN:
+		msg = " block quota exceeded.\r\n";
+		break;
+	}
+	tty_write_message(tty, msg);
+	tty_kref_put(tty);
+}
+#endif
+
+static void prepare_warning(struct ns_dquot_warn *warn, struct ns_dquot *dquot,
+			    int warntype)
+{
+	if (warning_issued(dquot, warntype))
+		return;
+
+	warn->w_type = warntype;
+	warn->w_ns = dquot->dq_ns;
+	warn->w_dq_id = dquot->dq_id;
+	warn->w_dq_type = dquot->dq_type;
+}
+
+/*
+ * Write warnings to the console and send warning messages over netlink.
+ * Note that this function can call into tty and networking code.
+ */
+static void flush_warnings(struct ns_dquot_warn *warn)
+{
+	int i;
+
+	for (i = 0; i < MAXQUOTAS; i++) {
+		if (warn[i].w_type == QUOTA_NL_NOWARN)
+			continue;
+#ifdef CONFIG_PRINT_QUOTA_WARNING
+#if 0
+		print_warning(&warn[i]);
+		quota_send_warning(warn[i].w_dq_type, warn[i].w_dq_id,
+				   warn[i].w_ns->s_dev, warn[i].w_type);
+#endif
+#endif
+	}
+}
+
+static struct ns_dquot *get_empty_ns_dquot(struct mnt_namespace *ns)
+{
+	return ns->ns_dqop->alloc_dquot(ns);
+}
+
+/* Find out or allocate a new dquot */
+static struct ns_dquot *ns_dqget(struct mnt_namespace *ns, unsigned int id,
+				 int type)
+{
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+	struct ns_dquot *dquot;
+
+	if (!dqinfo)
+		return NULL;
+
+	spin_lock(&dqinfo->dq_list_lock);
+	dquot = find_ns_dquot(ns, id, type);
+	if (!dquot) {
+		dquot = get_empty_ns_dquot(ns);
+		if (!dquot)
+			goto out_unlock;
+		INIT_HLIST_NODE(&dquot->dq_hash_node);
+		dquot->dq_ns = ns;
+		dquot->dq_id = id;
+		dquot->dq_type = type;
+		insert_ns_dquot_hash(dquot);
+	}
+
+out_unlock:
+	spin_unlock(&dqinfo->dq_list_lock);
+	return dquot;
+}
+
+/*
+ * FIXME:
+ * Below stuff regarding space calculations are all copied from general disk
+ * quota, need to refector them to reduce duplications maybe.
+ */
+static inline void ns_dquot_incr_inodes(struct ns_dquot *dquot, qsize_t number)
+{
+	dquot->dq_dqb.dqb_curinodes += number;
+}
+
+static inline void ns_dquot_resv_space(struct ns_dquot *dquot, qsize_t number)
+{
+	dquot->dq_dqb.dqb_rsvspace += number;
+}
+
+static inline void ns_dquot_incr_space(struct ns_dquot *dquot, qsize_t number)
+{
+	dquot->dq_dqb.dqb_curspace += number;
+}
+
+/* claim reserved quota space */
+static void ns_dquot_claim_reserved_space(struct ns_dquot *dquot,
+					  qsize_t number)
+{
+	if (dquot->dq_dqb.dqb_rsvspace < number) {
+		WARN_ON_ONCE(1);
+		number = dquot->dq_dqb.dqb_rsvspace;
+	}
+
+	dquot->dq_dqb.dqb_curspace += number;
+	dquot->dq_dqb.dqb_rsvspace -= number;
+}
+
+static inline void dquot_free_reserved_space(struct ns_dquot *dquot,
+					     qsize_t number)
+{
+	if (dquot->dq_dqb.dqb_rsvspace >= number)
+		dquot->dq_dqb.dqb_rsvspace -= number;
+	else {
+		WARN_ON_ONCE(1);
+		dquot->dq_dqb.dqb_rsvspace = 0;
+	}
+}
+
+static void ns_dquot_decr_inodes(struct ns_dquot *dquot, qsize_t number)
+{
+	struct ns_quota_info *dqinfo = dquot->dq_ns->ns_dqinfo;
+
+	if (dqinfo->dq_flags & DQUOT_NEGATIVE_USAGE ||
+	    dquot->dq_dqb.dqb_curinodes >= number)
+		dquot->dq_dqb.dqb_curinodes -= number;
+	else
+		dquot->dq_dqb.dqb_curinodes = 0;
+
+	if (dquot->dq_dqb.dqb_curinodes <= dquot->dq_dqb.dqb_isoftlimit)
+		dquot->dq_dqb.dqb_itime = (time_t)0;
+}
+
+static void ns_dquot_decr_space(struct ns_dquot *dquot, qsize_t number)
+{
+	struct ns_quota_info *dqinfo = dquot->dq_ns->ns_dqinfo;
+
+	if (dqinfo->dq_flags & DQUOT_NEGATIVE_USAGE ||
+	    dquot->dq_dqb.dqb_curspace >= number)
+		dquot->dq_dqb.dqb_curspace -= number;
+	else
+		dquot->dq_dqb.dqb_curspace = 0;
+	if (dquot->dq_dqb.dqb_curspace <= dquot->dq_dqb.dqb_bsoftlimit)
+		dquot->dq_dqb.dqb_btime = (time_t)0;
+}
+
+static int ns_check_idq(struct ns_dquot *dquot, qsize_t inodes,
+			struct ns_dquot_warn *warn)
+{
+	qsize_t newinodes = dquot->dq_dqb.dqb_curinodes + inodes;
+	struct mnt_namespace *ns = dquot->dq_ns;
+
+	if (!ns_has_quota_limit_enabled(ns, dquot->dq_type))
+		return 0;
+
+	if (dquot->dq_dqb.dqb_ihardlimit &&
+	    newinodes > dquot->dq_dqb.dqb_ihardlimit) {
+		prepare_warning(warn, dquot, QUOTA_NL_IHARDWARN);
+		return -EDQUOT;
+	}
+
+	if (dquot->dq_dqb.dqb_isoftlimit &&
+	    newinodes > dquot->dq_dqb.dqb_isoftlimit &&
+	    dquot->dq_dqb.dqb_itime &&
+	    get_seconds() >= dquot->dq_dqb.dqb_itime) {
+		prepare_warning(warn, dquot, QUOTA_NL_ISOFTLONGWARN);
+		return -EDQUOT;
+	}
+
+	if (dquot->dq_dqb.dqb_isoftlimit &&
+	    newinodes > dquot->dq_dqb.dqb_isoftlimit &&
+	    dquot->dq_dqb.dqb_itime == 0) {
+		prepare_warning(warn, dquot, QUOTA_NL_ISOFTWARN);
+		dquot->dq_dqb.dqb_itime = get_seconds() +
+			ns->ns_dqinfo->dqinfo[dquot->dq_type].dqi_igrace;
+	}
+
+	return 0;
+}
+
+static int ns_check_bdq(struct ns_dquot *dquot, qsize_t space,
+			struct ns_dquot_warn *warn)
+{
+	struct mnt_namespace *ns = dquot->dq_ns;
+	qsize_t tspace;
+
+	if (!ns_has_quota_limit_enabled(ns, dquot->dq_type))
+		return 0;
+
+	tspace = dquot->dq_dqb.dqb_curspace + dquot->dq_dqb.dqb_rsvspace
+		 + space;
+
+	if (dquot->dq_dqb.dqb_bhardlimit &&
+	    tspace > dquot->dq_dqb.dqb_bhardlimit) {
+		prepare_warning(warn, dquot, QUOTA_NL_BHARDWARN);
+		return -EDQUOT;
+	}
+
+	if (dquot->dq_dqb.dqb_bsoftlimit &&
+	    tspace > dquot->dq_dqb.dqb_bsoftlimit &&
+	    dquot->dq_dqb.dqb_btime &&
+	    get_seconds() >= dquot->dq_dqb.dqb_btime) {
+		prepare_warning(warn, dquot, QUOTA_NL_BHARDWARN);
+		return -EDQUOT;
+	}
+
+	if (dquot->dq_dqb.dqb_bsoftlimit &&
+	    tspace > dquot->dq_dqb.dqb_bsoftlimit &&
+	    dquot->dq_dqb.dqb_btime == 0) {
+		prepare_warning(warn, dquot, QUOTA_NL_BSOFTWARN);
+		dquot->dq_dqb.dqb_btime = get_seconds() +
+		ns->ns_dqinfo->dqinfo[dquot->dq_type].dqi_bgrace;
+		return -EDQUOT;
+	}
+
+	return 0;
+}
+
+static int __ns_dquot_alloc_space(const struct inode *inode, qsize_t number,
+				  int flags)
+{
+	int cnt, ret = 0;
+	struct ns_dquot_warn warn[MAXQUOTAS];
+	struct mnt_namespace *ns = current->nsproxy->mnt_ns;
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+	int reserve = flags & DQUOT_SPACE_RESERVE;
+
+	if (!dqinfo)
+		return 0;
+
+	if (!ns_any_quota_loaded(ns))
+		return 0;
+
+	spin_lock(&dqinfo->dq_list_lock);
+	for (cnt = 0; cnt < MAXQUOTAS; cnt++) {
+		unsigned int id;
+		struct ns_dquot *dquot;
+		warn[cnt].w_type = QUOTA_NL_NOWARN;
+
+		switch (cnt) {
+		case USRQUOTA:
+			id = inode->i_uid;
+			break;
+		case GRPQUOTA:
+			id = inode->i_gid;
+			break;
+		}
+		dquot = find_ns_dquot(ns, id, cnt);
+		if (!dquot)
+			continue;
+
+		ret = ns_check_bdq(dquot, number, &warn[cnt]);
+		if (ret && !(flags & DQUOT_SPACE_NOFAIL))
+			goto out_flush_warn;
+
+		spin_lock(&dqinfo->dq_data_lock);
+		if (reserve)
+			ns_dquot_resv_space(dquot, number);
+		else
+			ns_dquot_incr_space(dquot, number);
+		spin_unlock(&dqinfo->dq_data_lock);
+	}
+
+out_flush_warn:
+	spin_unlock(&dqinfo->dq_list_lock);
+	flush_warnings(warn);
+	return ret;
+}
+
+/* Exported routine for file system disk space quota checking */
+int ns_dquot_alloc_block(struct inode *inode, qsize_t nr)
+{
+	return __ns_dquot_alloc_space(inode, nr << inode->i_blkbits,
+				      DQUOT_SPACE_WARN);
+}
+EXPORT_SYMBOL(ns_dquot_alloc_block);
+
+static void ns_dquot_alloc_space_nofail(struct inode *inode, qsize_t nr)
+{
+	__ns_dquot_alloc_space(inode, nr, DQUOT_SPACE_WARN|DQUOT_SPACE_NOFAIL);
+}
+
+void ns_dquot_alloc_block_nofail(struct inode *inode, qsize_t nr)
+{
+	ns_dquot_alloc_space_nofail(inode, nr << inode->i_blkbits);
+}
+EXPORT_SYMBOL(ns_dquot_alloc_block_nofail);
+
+int ns_dquot_reserve_block(struct inode *inode, qsize_t nr)
+{
+	return __ns_dquot_alloc_space(inode, nr << inode->i_blkbits,
+				DQUOT_SPACE_WARN|DQUOT_SPACE_RESERVE);
+}
+EXPORT_SYMBOL(ns_dquot_reserve_block);
+
+void ns_dquot_claim_block(struct inode *inode, qsize_t nr)
+{
+	int cnt;
+	struct mnt_namespace *ns = current->nsproxy->mnt_ns;
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+
+	if (!dqinfo)
+		return;
+
+	if (!ns_any_quota_loaded(ns))
+		return;
+
+	spin_lock(&dqinfo->dq_list_lock);
+	for (cnt = 0; cnt < MAXQUOTAS; cnt++) {
+		unsigned int id;
+		struct ns_dquot *dquot;
+
+		switch (cnt) {
+		case USRQUOTA:
+			id = inode->i_uid;
+			break;
+		case GRPQUOTA:
+			id = inode->i_gid;
+			break;
+		}
+		dquot = find_ns_dquot(ns, id, cnt);
+		if (!dquot)
+			continue;
+
+		spin_lock(&dqinfo->dq_data_lock);
+		ns_dquot_claim_reserved_space(dquot, nr << inode->i_blkbits);
+		spin_unlock(&dqinfo->dq_data_lock);
+	}
+
+	spin_unlock(&dqinfo->dq_list_lock);
+}
+EXPORT_SYMBOL(ns_dquot_claim_block);
+
+/* This operation can block, but only after everything is updated */
+int ns_dquot_alloc_inode(const struct inode *inode)
+{
+	int cnt, ret = 0;
+	struct ns_dquot_warn warn[MAXQUOTAS];
+	struct mnt_namespace *ns = current->nsproxy->mnt_ns;
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+
+	if (!dqinfo)
+		return 0;
+
+	if (!ns_any_quota_loaded(ns))
+		return 0;
+
+	spin_lock(&dqinfo->dq_list_lock);
+	for (cnt = 0; cnt < MAXQUOTAS; cnt++) {
+		struct ns_dquot *dquot;
+		unsigned int id;
+		warn[cnt].w_type = QUOTA_NL_NOWARN;
+
+		switch (cnt) {
+		case USRQUOTA:
+			id = inode->i_uid;
+			break;
+		case GRPQUOTA:
+			id = inode->i_gid;
+			break;
+		}
+
+		dquot = find_ns_dquot(ns, id, cnt);
+		if (!dquot)
+			continue;
+
+		ret = ns_check_idq(dquot, 1, &warn[cnt]);
+		if (ret)
+			goto over_quota;
+
+		spin_lock(&dqinfo->dq_data_lock);
+		ns_dquot_incr_inodes(dquot, 1);
+		spin_unlock(&dqinfo->dq_data_lock);
+	}
+
+over_quota:
+	spin_unlock(&dqinfo->dq_list_lock);
+	flush_warnings(warn);
+	return ret;
+}
+EXPORT_SYMBOL(ns_dquot_alloc_inode);
+
+static int ns_info_bdq_free(struct ns_dquot *dquot, qsize_t space)
+{
+	struct mem_dqblk *dq_dqb = &dquot->dq_dqb;
+
+	if (dq_dqb->dqb_curspace <= dq_dqb->dqb_bsoftlimit)
+		return QUOTA_NL_NOWARN;
+
+	if (dq_dqb->dqb_curspace - space <= dq_dqb->dqb_bsoftlimit)
+		return QUOTA_NL_BSOFTBELOW;
+
+	if (dq_dqb->dqb_curspace >= dq_dqb->dqb_bhardlimit &&
+	    dq_dqb->dqb_curspace - space < dq_dqb->dqb_bhardlimit)
+		return QUOTA_NL_BHARDBELOW;
+
+	return QUOTA_NL_NOWARN;
+}
+
+static void __ns_dquot_free_space(const struct inode *inode, qsize_t number,
+				  int flags)
+{
+	unsigned int cnt;
+	struct ns_dquot_warn warn[MAXQUOTAS];
+	struct mnt_namespace *ns = current->nsproxy->mnt_ns;
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+
+	dqinfo = ns->ns_dqinfo;
+	if (!dqinfo)
+		return;
+
+	if (!ns_any_quota_loaded(ns))
+		return;
+
+	spin_lock(&dqinfo->dq_list_lock);
+	for (cnt = 0; cnt < MAXQUOTAS; cnt++) {
+		int wtype;
+		unsigned int id;
+		struct ns_dquot *dquot;
+		warn[cnt].w_type = QUOTA_NL_NOWARN;
+
+		switch (cnt) {
+		case USRQUOTA:
+			id = inode->i_uid;
+			break;
+		case GRPQUOTA:
+			id = inode->i_gid;
+			break;
+		}
+
+		dquot = find_ns_dquot(ns, id, cnt);
+		if (!dquot)
+			continue;
+
+		wtype = ns_info_bdq_free(dquot, number);
+		if (wtype != QUOTA_NL_NOWARN)
+			prepare_warning(&warn[cnt], dquot, wtype);
+		spin_lock(&dqinfo->dq_data_lock);
+		ns_dquot_decr_space(dquot, number);
+		spin_unlock(&dqinfo->dq_data_lock);
+	}
+	spin_unlock(&dqinfo->dq_list_lock);
+	flush_warnings(warn);
+}
+
+void ns_dquot_free_block(struct inode *inode, qsize_t nr)
+{
+	__ns_dquot_free_space(inode, nr << inode->i_blkbits, 0);
+}
+EXPORT_SYMBOL(ns_dquot_free_block);
+
+void ns_dquot_release_reservation_block(struct inode *inode, qsize_t nr)
+{
+	__ns_dquot_free_space(inode, nr << inode->i_blkbits,
+			      DQUOT_SPACE_RESERVE);
+}
+EXPORT_SYMBOL(ns_dquot_release_reservation_block);
+
+static int ns_info_idq_free(struct ns_dquot *dquot, qsize_t inodes)
+{
+	struct mem_dqblk *dq_dqb = &dquot->dq_dqb;
+	qsize_t newinodes;
+
+	if (dq_dqb->dqb_curinodes <= dq_dqb->dqb_isoftlimit ||
+	    !ns_has_quota_limit_enabled(dquot->dq_ns, dquot->dq_type))
+		return QUOTA_NL_NOWARN;
+
+	newinodes = dq_dqb->dqb_curinodes - inodes;
+	if (newinodes <= dq_dqb->dqb_isoftlimit)
+		return QUOTA_NL_ISOFTBELOW;
+
+	if (dq_dqb->dqb_curinodes >= dq_dqb->dqb_ihardlimit &&
+	    newinodes < dq_dqb->dqb_ihardlimit)
+		return QUOTA_NL_IHARDBELOW;
+
+	return QUOTA_NL_NOWARN;
+}
+
+/* Exported routine for inode removing. */
+void ns_dquot_free_inode(const struct inode *inode)
+{
+	unsigned int cnt;
+	struct ns_dquot_warn warn[MAXQUOTAS];
+	struct mnt_namespace *ns = current->nsproxy->mnt_ns;
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+
+	if (!dqinfo)
+		return;
+
+	if (!ns_any_quota_loaded(ns))
+		return;
+
+	spin_lock(&dqinfo->dq_list_lock);
+	for (cnt = 0; cnt < MAXQUOTAS; cnt++) {
+		unsigned int id;
+		struct ns_dquot *dquot;
+		int wtype;
+
+		switch (cnt) {
+		case USRQUOTA:
+			id = inode->i_uid;
+			break;
+		case GRPQUOTA:
+			id = inode->i_gid;
+			break;
+		}
+
+		dquot = find_ns_dquot(ns, id, cnt);
+		if (!dquot)
+			continue;
+
+		warn[cnt].w_type = QUOTA_NL_NOWARN;
+		wtype = ns_info_idq_free(dquot, 1);
+		if (wtype != QUOTA_NL_NOWARN)
+			prepare_warning(&warn[cnt], dquot, wtype);
+		spin_lock(&dqinfo->dq_data_lock);
+		ns_dquot_decr_inodes(dquot, 1);
+		spin_unlock(&dqinfo->dq_data_lock);
+	}
+	spin_unlock(&dqinfo->dq_list_lock);
+}
+EXPORT_SYMBOL(ns_dquot_free_inode);
+
+/*
+ * Definitions of diskquota operations.
+ */
+const struct ns_dquot_ops ns_dquot_operations = {
+	.alloc_dquot	= ns_dquot_alloc,
+	.destroy_dquot	= ns_dquot_destroy,
+};
+
+/*
+ * Transfer the number of inode and blocks from one diskquota to an other.
+ * On success, dquot references in transfer_to are consumed and references
+ * to original dquots that need to be released are placed there. On failure,
+ * references are kept untouched.
+ *
+ * This operation can block, but only after everything is updated
+ * A transaction must be started when entering this function.
+ */
+static int __ns_dquot_transfer(struct mnt_namespace *ns, struct inode *inode,
+			       struct ns_dquot **transfer_to)
+{
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+	struct ns_dquot *transfer_from[MAXQUOTAS] = {};
+	struct ns_dquot_warn warn[MAXQUOTAS];
+	char is_valid[MAXQUOTAS] = {};
+	int cnt, ret = 0;
+	qsize_t space;
+
+	spin_lock(&dqinfo->dq_data_lock);
+	space = inode_get_bytes(inode);
+
+	/* Build the transfer_from list and check the limits */
+	for (cnt = 0; cnt < MAXQUOTAS; cnt++) {
+		unsigned int id;
+		warn[cnt].w_type = QUOTA_NL_NOWARN;
+		/*
+		 * Skip changes for same uid or gid or for turned off
+		 * quota-type.
+		 */
+		if (!transfer_to[cnt])
+			continue;
+
+		/* Avoid races with quotaoff() */
+		if (!ns_has_quota_loaded(ns, cnt))
+			continue;
+
+		is_valid[cnt] = 1;
+		switch (cnt) {
+		case USRQUOTA:
+			id = inode->i_uid;
+			break;
+		case GRPQUOTA:
+			id = inode->i_gid;
+			break;
+		}
+
+		transfer_from[cnt] = find_ns_dquot(ns, id, cnt);
+		ret = ns_check_idq(transfer_to[cnt], 1, &warn[cnt]);
+		if (ret)
+			goto over_quota;
+
+		ret = ns_check_bdq(transfer_to[cnt], space, &warn[cnt]);
+		if (ret)
+			goto over_quota;
+	}
+
+	/*
+	 * Finally perform the needed transfer from transfer_from to
+	 * transfer_to.
+	 */
+	for (cnt = 0; cnt < MAXQUOTAS; cnt++) {
+		if (!is_valid[cnt])
+			continue;
+
+		/*
+		 * Due to IO error we might not have transfer_from[]
+		 * structure.
+		 */
+		if (transfer_from[cnt]) {
+			ns_dquot_decr_inodes(transfer_from[cnt], 1);
+			ns_dquot_decr_space(transfer_from[cnt], space);
+		}
+
+		ns_dquot_incr_inodes(transfer_to[cnt], 1);
+		ns_dquot_incr_space(transfer_to[cnt], space);
+	}
+
+over_quota:
+	spin_unlock(&dqinfo->dq_data_lock);
+	return ret;
+}
+
+/*
+ * Wrapper for transferring ownership of an inode for uid/gid only
+ * Called from FSXXX_setattr()
+ */
+int ns_dquot_transfer(struct inode *inode, struct iattr *iattr)
+{
+	struct mnt_namespace *ns = current->nsproxy->mnt_ns;
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+	struct ns_dquot *transfer_to[MAXQUOTAS] = {};
+	int ret = 0;
+
+	if (!dqinfo)
+		return ret;
+
+	if (!ns_any_quota_loaded(ns))
+		return ret;
+
+	spin_lock(&dqinfo->dq_list_lock);
+	if (iattr->ia_valid & ATTR_UID && iattr->ia_uid != inode->i_uid)
+		transfer_to[USRQUOTA] = find_ns_dquot(ns, iattr->ia_uid,
+						      USRQUOTA);
+	if (iattr->ia_valid & ATTR_GID && iattr->ia_gid != inode->i_gid)
+		transfer_to[GRPQUOTA] = find_ns_dquot(ns, iattr->ia_gid,
+						      GRPQUOTA);
+
+	ret = __ns_dquot_transfer(ns, inode, transfer_to);
+	spin_unlock(&dqinfo->dq_list_lock);
+
+	return ret;
+}
+EXPORT_SYMBOL(ns_dquot_transfer);
+
+unsigned int ns_dquot_getfmt(struct mnt_namespace *ns, int type)
+{
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+
+	if (!dqinfo || !ns_has_quota_loaded(ns, type))
+		return -ESRCH;
+
+	return QFMT_NS;
+}
+
+/*
+ * Activate disk quota on a particular namespace.
+ */
+static int ns_dquot_quota_on(struct mnt_namespace *ns, int type)
+{
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+	unsigned int flags;
+	int ret = 0;
+
+	if (!dqinfo)
+		return -ENOSYS;
+
+	mutex_lock(&dqinfo->dqonoff_mutex);
+	if (ns_has_quota_limit_enabled(ns, type)) {
+		ret = -EBUSY;
+		goto out_unlock;
+	}
+
+	/* Both disk quota usage and limits should be turn on */
+	flags = DQUOT_USAGE_ENABLED | DQUOT_LIMITS_ENABLED;
+	spin_lock(&ns_dq_state_lock);
+	dqinfo->dq_flags |= dquot_state_flag(flags, type);
+	spin_unlock(&ns_dq_state_lock);
+
+out_unlock:
+	mutex_unlock(&dqinfo->dqonoff_mutex);
+	return ret;
+}
+
+static int ns_dquot_disable(struct mnt_namespace *ns, int type,
+			    unsigned int flags)
+{
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+	int cnt;
+
+	if (!dqinfo)
+		return -ENOSYS;
+
+	mutex_lock(&dqinfo->dqonoff_mutex);
+	if (!ns_any_quota_loaded(ns))
+		goto out_unlock;
+
+	for (cnt = 0; cnt < MAXQUOTAS; cnt++) {
+		if (type != -1 && cnt != type)
+			continue;
+		if (!ns_has_quota_loaded(ns, cnt))
+			continue;
+
+		spin_lock(&ns_dq_state_lock);
+		dqinfo->dq_flags &= ~dquot_state_flag(flags, cnt);
+		spin_unlock(&ns_dq_state_lock);
+	}
+
+out_unlock:
+	mutex_unlock(&dqinfo->dqonoff_mutex);
+	return 0;
+}
+
+static int ns_dquot_quota_off(struct mnt_namespace *ns, int type)
+{
+	return ns_dquot_disable(ns, type,
+				DQUOT_USAGE_ENABLED | DQUOT_LIMITS_ENABLED);
+}
+
+/*
+ * FIXME:
+ * Below two routines are copied from general quota, they can be
+ * can be shared.
+ */
+static inline qsize_t qbtos(qsize_t blocks)
+{
+	return blocks << QIF_DQBLKSIZE_BITS;
+}
+
+static inline qsize_t stoqb(qsize_t space)
+{
+	return (space + QIF_DQBLKSIZE - 1) >> QIF_DQBLKSIZE_BITS;
+}
+
+/* Generic routine for getting common part of quota structure */
+static void do_get_ns_dqblk(struct ns_dquot *dquot, struct fs_disk_quota *di)
+{
+	struct ns_quota_info *dqinfo = dquot->dq_ns->ns_dqinfo;
+	struct mem_dqblk *dm = &dquot->dq_dqb;
+
+	memset(di, 0, sizeof(*di));
+	di->d_version = FS_DQUOT_VERSION;
+	di->d_flags = dquot->dq_type == USRQUOTA ?
+					FS_USER_QUOTA : FS_GROUP_QUOTA;
+	di->d_id = dquot->dq_id;
+
+	spin_lock(&dqinfo->dq_data_lock);
+	di->d_blk_hardlimit = stoqb(dm->dqb_bhardlimit);
+	di->d_blk_softlimit = stoqb(dm->dqb_bsoftlimit);
+	di->d_ino_hardlimit = dm->dqb_ihardlimit;
+	di->d_ino_softlimit = dm->dqb_isoftlimit;
+	di->d_bcount = dm->dqb_curspace + dm->dqb_rsvspace;
+	di->d_icount = dm->dqb_curinodes;
+	di->d_btimer = dm->dqb_btime;
+	di->d_itimer = dm->dqb_itime;
+	spin_unlock(&dqinfo->dq_data_lock);
+}
+
+static int ns_dquot_get_dqblk(struct mnt_namespace *ns, int type, qid_t id,
+			      struct fs_disk_quota *di)
+{
+	struct ns_dquot *dquot;
+
+	dquot = ns_dqget(ns, id, type);
+	if (!dquot)
+		return -ESRCH;
+
+	do_get_ns_dqblk(dquot, di);
+	return 0;
+}
+
+static int do_set_ns_dqblk(struct ns_dquot *dquot, struct fs_disk_quota *di)
+{
+	struct ns_quota_info *dqinfo = dquot->dq_ns->ns_dqinfo;
+	struct mem_dqblk *dm = &dquot->dq_dqb;
+
+	if (di->d_fieldmask & ~VFS_FS_DQ_MASK)
+		return -EINVAL;
+
+	spin_lock(&dqinfo->dq_data_lock);
+	if (di->d_fieldmask & FS_DQ_BCOUNT)
+		dm->dqb_curspace = di->d_bcount - dm->dqb_rsvspace;
+
+	if (di->d_fieldmask & FS_DQ_BSOFT)
+		dm->dqb_bsoftlimit = qbtos(di->d_blk_softlimit);
+
+	if (di->d_fieldmask & FS_DQ_BHARD)
+		dm->dqb_bhardlimit = qbtos(di->d_blk_hardlimit);
+
+	if (di->d_fieldmask & FS_DQ_ICOUNT)
+		dm->dqb_curinodes = di->d_icount;
+
+	if (di->d_fieldmask & FS_DQ_ISOFT)
+		dm->dqb_isoftlimit = di->d_ino_softlimit;
+
+	if (di->d_fieldmask & FS_DQ_IHARD)
+		dm->dqb_ihardlimit = di->d_ino_hardlimit;
+
+	if (di->d_fieldmask & FS_DQ_BTIMER)
+		dm->dqb_btime = di->d_btimer;
+
+	if (di->d_fieldmask & FS_DQ_ITIMER)
+		dm->dqb_itime = di->d_itimer;
+	spin_unlock(&dqinfo->dq_data_lock);
+
+	return 0;
+}
+
+static int ns_dquot_set_dqblk(struct mnt_namespace *ns, int type,
+			      qid_t id, struct fs_disk_quota *di)
+{
+	struct ns_dquot *dquot;
+
+	dquot = ns_dqget(ns, id, type);
+	if (!dquot)
+		return -ESRCH;
+
+	return do_set_ns_dqblk(dquot, di);
+}
+
+static int ns_dquot_get_dqinfo(struct mnt_namespace *ns, int type,
+			       struct if_dqinfo *ii)
+{
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+	struct ns_mem_dqinfo *mi;
+	int ret = 0;
+
+	if (!dqinfo)
+		return 0;
+
+	mutex_lock(&dqinfo->dqonoff_mutex);
+	if (!ns_has_quota_active(ns, type)) {
+		ret = -ESRCH;
+		goto out_unlock;
+	}
+
+	mi = dqinfo->dqinfo + type;
+	spin_lock(&dqinfo->dq_data_lock);
+	ii->dqi_bgrace = mi->dqi_bgrace;
+	ii->dqi_igrace = mi->dqi_bgrace;
+	ii->dqi_flags = mi->dqi_flags & DQF_GETINFO_MASK;
+	ii->dqi_valid = IIF_ALL;
+	spin_unlock(&dqinfo->dq_data_lock);
+
+out_unlock:
+	mutex_unlock(&dqinfo->dqonoff_mutex);
+	return ret;
+}
+
+static int ns_dquot_set_dqinfo(struct mnt_namespace *ns, int type,
+			       struct if_dqinfo *ii)
+{
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+	struct ns_mem_dqinfo *mi;
+	int ret = 0;
+
+	if (!dqinfo)
+		return 0;
+
+	mutex_lock(&dqinfo->dqonoff_mutex);
+	if (!ns_has_quota_loaded(ns, type)) {
+		ret = -ESRCH;
+		goto out;
+	}
+
+	mi = dqinfo->dqinfo + type;
+	spin_lock(&dqinfo->dq_data_lock);
+	if (ii->dqi_valid & IIF_BGRACE)
+		mi->dqi_bgrace = ii->dqi_bgrace;
+	if (ii->dqi_valid & IIF_IGRACE)
+		mi->dqi_igrace = ii->dqi_igrace;
+	if (ii->dqi_valid & IIF_FLAGS)
+		mi->dqi_flags = (mi->dqi_flags & ~DQF_SETINFO_MASK) |
+				(ii->dqi_flags & DQF_SETINFO_MASK);
+	spin_unlock(&dqinfo->dq_data_lock);
+
+out:
+	mutex_unlock(&dqinfo->dqonoff_mutex);
+	return ret;
+}
+
+const struct ns_quotactl_ops ns_quotactl_operations = {
+	.quota_on	= ns_dquot_quota_on,
+	.quota_off	= ns_dquot_quota_off,
+	.get_dqblk	= ns_dquot_get_dqblk,
+	.set_dqblk	= ns_dquot_set_dqblk,
+	.get_info	= ns_dquot_get_dqinfo,
+	.set_info	= ns_dquot_set_dqinfo,
+};
+
+int ns_dqinfo_init(struct mnt_namespace *ns)
+{
+	struct ns_quota_info *dqinfo;
+	char tmp[16];
+	int i;
+
+	ns->ns_dqinfo = kmalloc(sizeof(struct ns_quota_info), GFP_NOFS);
+	if (!ns->ns_dqinfo)
+		return -ENOMEM;
+
+	dqinfo = ns->ns_dqinfo;
+	dqinfo->dq_flags = 0;	/* Disk quota is disabled by default */
+	mutex_init(&dqinfo->dqonoff_mutex);
+	spin_lock_init(&dqinfo->dq_list_lock);
+	spin_lock_init(&dqinfo->dq_data_lock);
+
+	/*
+	 * Currently, using "ns_dquot_" combine with operation process id
+	 * to indentify dquot cache per mount namespace.
+	 * FIXME:
+	 * Need to examine a reasonable identifier for that.
+	 */
+	snprintf(tmp, sizeof(tmp), "ns_dquot_%d", current->pid);
+	dqinfo->dquot_cachep = kmem_cache_create(tmp, sizeof(struct ns_dquot),
+						 0, SLAB_PANIC, NULL);
+	if (!dqinfo->dquot_cachep) {
+		kfree(dqinfo);
+		return -ENOMEM;
+	}
+
+	for (i = 0; i < NS_DQHASH_SZ; ++i) {
+		INIT_HLIST_HEAD(dqinfo->u_dquots + i);
+		INIT_HLIST_HEAD(dqinfo->g_dquots + i);
+	}
+
+	for (i = 0; i < MAXQUOTAS; i++) {
+		/* Used space is stored as unsigned 64-bit value, 2^64 - 1 */
+		dqinfo->dqinfo[i].dqi_maxblimit = 0xffffffffffffffffULL;
+		dqinfo->dqinfo[i].dqi_maxilimit = 0xffffffffffffffffULL;
+
+		/* Grace time is stored as (7*24*60*60) 1 week */
+		dqinfo->dqinfo[i].dqi_igrace = NS_MAX_IQ_TIME;
+		dqinfo->dqinfo[i].dqi_bgrace = NS_MAX_DQ_TIME;
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL(ns_dqinfo_init);
+
+/*
+ * Free the all allocated disk quotas if a mount namespace with disk
+ * quota enabled will be destroyed.
+ */
+void ns_dqinfo_destroy(struct mnt_namespace *ns)
+{
+	struct ns_quota_info *dqinfo = ns->ns_dqinfo;
+	int i;
+
+	if (!dqinfo)
+		return;
+
+	for (i = 0; i < NS_DQHASH_SZ; ++i) {
+		__remove_dq_hash_list(&dqinfo->u_dquots[i]);
+		__remove_dq_hash_list(&dqinfo->g_dquots[i]);
+	}
+
+	kmem_cache_destroy(dqinfo->dquot_cachep);
+	kfree(dqinfo);
+}
+EXPORT_SYMBOL(ns_dqinfo_destroy);
+
+/*
+ * FIXME:
+ * Need printing out debug information like current container
+ * disk quota VERSION?
+ */
+static int __init ns_dquot_init(void)
+{
+	return 0;
+}
+
+static void __exit ns_dquot_exit(void)
+{
+	return;
+}
+
+module_init(ns_dquot_init);
+module_exit(ns_dquot_exit);
diff --git a/fs/ns_quotaops.h b/fs/ns_quotaops.h
new file mode 100644
index 0000000..6eed233
--- /dev/null
+++ b/fs/ns_quotaops.h
@@ -0,0 +1,72 @@
+#ifndef _LINUX_NS_QUOTAOPS_
+#define _LINUX_NS_QUOTAOPS_
+
+#include <linux/fs.h>
+
+#ifdef CONFIG_NS_QUOTA
+
+extern int do_quotactl_for_container(const char __user *);
+extern int do_container_quotactl(int, int, qid_t, void __user *);
+
+int ns_dquot_alloc_inode(const struct inode *inode);
+void ns_dquot_free_inode(const struct inode *inode);
+int ns_dquot_alloc_block(const struct inode *inode, qsize_t nr);
+void ns_dquot_alloc_block_nofail(const struct inode *inode, qsize_t nr);
+void ns_dquot_free_block(const struct inode *inode, qsize_t nr);
+int ns_dquot_transfer(struct inode *inode, struct iattr *iattr);
+int ns_dquot_reserve_block(struct inode *inode, qsize_t nr);
+void ns_dquot_claim_block(struct inode *inode, qsize_t nr);
+void ns_dquot_release_reservation_block(struct inode *inode, qsize_t nr);
+
+/*
+ * Operations supported for mount namespace disk quotas.
+ */
+extern const struct ns_quotactl_ops ns_quotactl_operations;
+extern const struct ns_dquot_ops ns_dquot_operations;
+
+#else
+
+static inline int ns_dquot_alloc_inode(const struct inode *inode)
+{
+	return 0;
+}
+
+static inline void ns_dquot_free_inode(const struct inode *inode)
+{
+}
+
+static inline void ns_dquot_alloc_block_nofail(const struct inode *inode,
+					       qsize_t nr)
+{
+}
+
+static int ns_dquot_alloc_block(const struct inode *inode, qsize_t nr)
+{
+	return 0;
+}
+
+static void ns_dquot_free_block(const struct inode *inode, qsize_t nr)
+{
+}
+
+static int ns_dquot_transfer(struct inode *inode, struct iattr *iattr)
+{
+	return 0;
+}
+
+static void ns_dquot_claim_block(struct inode *inode, qsize_t nr)
+{
+}
+
+static void ns_dquot_release_reservation_block(struct inode *inode, qsize_t nr)
+{
+}
+
+static int ns_dquot_reserve_block(struct inode *inode, qsize_t nr)
+{
+	return 0;
+}
+
+#endif /* __CONFIG_NS_QUOTA__ */
+
+#endif /* _LINUX_NS_QUOTAOPS_ */
-- 
1.7.9

[PATCH 07/12] container quota: add quota control source file.

[jeff.liu-QHcLZuEGTsvQT0dZR+AlfA]

Add container disk quota control source file.

Signed-off-by: Jie Liu <jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 fs/ns_quota.c |  261 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 261 insertions(+), 0 deletions(-)
 create mode 100644 fs/ns_quota.c

diff --git a/fs/ns_quota.c b/fs/ns_quota.c
new file mode 100644
index 0000000..9d24041
--- /dev/null
+++ b/fs/ns_quota.c
@@ -0,0 +1,261 @@
+#include <linux/fs.h>
+#include <linux/slab.h>
+#include <asm/current.h>
+#include <linux/uaccess.h>
+#include <linux/kernel.h>
+#include <linux/security.h>
+#include <linux/syscalls.h>
+#include <linux/capability.h>
+#include <linux/types.h>
+#include <linux/writeback.h>
+#include <linux/nsproxy.h>
+#include <linux/mnt_namespace.h>
+#include "mount.h"
+
+/*
+ * The corresponding device of "/" and file system type is "rootfs"
+ * if quotactl(2) is invoked from a container guest.
+ */
+static int is_container_rootfs(const char __user *special)
+{
+	int ret;
+	char *tmp = getname(special);
+
+	if (IS_ERR(tmp))
+		return PTR_ERR(tmp);
+
+	ret = strcmp(tmp, "rootfs");
+	putname(tmp);
+
+	return !ret;
+}
+
+/*
+ * Currently, to ensure quotactl(2) is invoked from a container VM or a
+ * cloned mount namespace created through unshare(1), I do check that the
+ * input dev is "rootfs" or the current pid namespace is not the initial
+ * one.  Is that sounds stupid enough? :(
+ *
+ * FIXME:
+ * Need to find out a reasonable approach to examine whether perform
+ * container disk quota or not.
+ * Some of my thoughs were shown as following:
+ * 1. Define a couple of pariticular NS_QUOTAON/NS_QUOTAOFF/NS_QGETINFO, etc.
+ *    do container disk quota if they are presented.
+ * 2. Maybe people prefer to make use of container disk quota through
+ *    unshare(1) combine with cgroups, and they even don't want run
+ *    quotacheck(8) in this case, they just want to limit those quota stuff
+ *    in a strightford way without disk usage pre-checkup, something like:
+ *    turn quota on a particular mount namespace, set the quota limits per
+ *    their requirements, stop further storage operations once over quota
+ *    limits.  And also, the quota limits can across different storage if
+ *    the underlying file systems are running with container quota enabled.
+ */
+int do_quotactl_for_container(const char __user *special)
+{
+	return (is_container_rootfs(special) ||
+		current->nsproxy->pid_ns != &init_pid_ns) ? 1 : 0;
+}
+
+/*
+ * FIXME: find out a way to solve mount namespace security/cap verfication.
+ * Something like: ns_capable(current->nsproxy->mnt_ns, CAP_XXXX)?
+ */
+static int check_ns_quotactl_permission(struct mnt_namespace *ns,
+					int type, int cmd, qid_t id)
+{
+	switch (cmd) {
+	/* these commands do not require any special privilegues */
+	case Q_GETFMT:
+	case Q_GETINFO:
+		break;
+	/* allow to query information for dquots we "own" */
+	case Q_GETQUOTA:
+		if ((type == USRQUOTA && current_euid() == id) ||
+		    (type == GRPQUOTA && in_egroup_p(id)))
+			break;
+		/*fallthrough*/
+	default:
+		if (!capable(CAP_SYS_ADMIN))
+			return -EPERM;
+	}
+
+	return 0;
+}
+
+/*
+ * FIXME:
+ * The following helpers are copied from general quota, they can be
+ * shared actally.
+ */
+static void copy_to_if_dqblk(struct if_dqblk *dst, struct fs_disk_quota *src)
+{
+	dst->dqb_bhardlimit = src->d_blk_hardlimit;
+	dst->dqb_bsoftlimit = src->d_blk_softlimit;
+	dst->dqb_curspace = src->d_bcount;
+	dst->dqb_ihardlimit = src->d_ino_hardlimit;
+	dst->dqb_isoftlimit = src->d_ino_softlimit;
+	dst->dqb_curinodes = src->d_icount;
+	dst->dqb_btime = src->d_btimer;
+	dst->dqb_itime = src->d_itimer;
+	dst->dqb_valid = QIF_ALL;
+}
+
+static void copy_from_if_dqblk(struct fs_disk_quota *dst, struct if_dqblk *src)
+{
+	dst->d_blk_hardlimit = src->dqb_bhardlimit;
+	dst->d_blk_softlimit  = src->dqb_bsoftlimit;
+	dst->d_bcount = src->dqb_curspace;
+	dst->d_ino_hardlimit = src->dqb_ihardlimit;
+	dst->d_ino_softlimit = src->dqb_isoftlimit;
+	dst->d_icount = src->dqb_curinodes;
+	dst->d_btimer = src->dqb_btime;
+	dst->d_itimer = src->dqb_itime;
+
+	dst->d_fieldmask = 0;
+	if (src->dqb_valid & QIF_BLIMITS)
+		dst->d_fieldmask |= FS_DQ_BSOFT | FS_DQ_BHARD;
+	if (src->dqb_valid & QIF_SPACE)
+		dst->d_fieldmask |= FS_DQ_BCOUNT;
+	if (src->dqb_valid & QIF_ILIMITS)
+		dst->d_fieldmask |= FS_DQ_ISOFT | FS_DQ_IHARD;
+	if (src->dqb_valid & QIF_INODES)
+		dst->d_fieldmask |= FS_DQ_ICOUNT;
+	if (src->dqb_valid & QIF_BTIME)
+		dst->d_fieldmask |= FS_DQ_BTIMER;
+	if (src->dqb_valid & QIF_ITIME)
+		dst->d_fieldmask |= FS_DQ_ITIMER;
+}
+
+static int ns_quota_on(struct mnt_namespace *ns, int type)
+{
+	return ns->ns_qcop->quota_on(ns, type);
+}
+
+static int ns_quota_off(struct mnt_namespace *ns, int type)
+{
+	return ns->ns_qcop->quota_off(ns, type);
+}
+
+static int ns_quota_getinfo(struct mnt_namespace *ns, int type,
+			    void __user *addr)
+{
+	struct if_dqinfo info;
+	int ret;
+
+	ret = ns->ns_qcop->get_info(ns, type, &info);
+	if (!ret && copy_to_user(addr, &info, sizeof(info)))
+		return -EFAULT;
+
+	return ret;
+}
+
+static int ns_quota_setinfo(struct mnt_namespace *ns, int type,
+			    void __user *addr)
+{
+	struct if_dqinfo info;
+
+	if (copy_from_user(&info, addr, sizeof(info)))
+		return -EFAULT;
+
+	return ns->ns_qcop->set_info(ns, type, &info);
+}
+
+static int ns_quota_getquota(struct mnt_namespace *ns, int type,
+			     qid_t id, void __user *addr)
+{
+	struct fs_disk_quota fdq;
+	struct if_dqblk idq;
+	int ret;
+
+	ret = ns->ns_qcop->get_dqblk(ns, type, id, &fdq);
+	if (ret)
+		return ret;
+
+	copy_to_if_dqblk(&idq, &fdq);
+	if (copy_to_user(addr, &idq, sizeof(idq)))
+		return -EFAULT;
+
+	return 0;
+}
+
+static int ns_quota_setquota(struct mnt_namespace *ns, int type, qid_t id,
+			     void __user *addr)
+{
+	struct fs_disk_quota fdq;
+	struct if_dqblk idq;
+
+	if (copy_from_user(&idq, addr, sizeof(idq)))
+		return -EFAULT;
+
+	copy_from_if_dqblk(&fdq, &idq);
+	return ns->ns_qcop->set_dqblk(ns, type, id, &fdq);
+}
+
+static int ns_quota_getfmt(struct mnt_namespace *ns, int type,
+			   void __user *addr)
+{
+	__u32 fmt;
+
+	fmt = ns_dquot_getfmt(ns, type);
+	if (!fmt)
+		return fmt;
+
+	if (copy_to_user(addr, &fmt, sizeof(fmt)))
+		return -EFAULT;
+	return 0;
+}
+
+/* Copy parameters and call proper function */
+int do_container_quotactl(int type, int cmd, qid_t id, void __user *addr)
+{
+	struct mnt_namespace *ns = current->nsproxy->mnt_ns;
+	int ret = 0;
+
+	if (type >= (XQM_COMMAND(cmd) ? XQM_MAXQUOTAS : MAXQUOTAS))
+		return -EINVAL;
+
+	lock_mnt_ns(ns);
+	ret = check_ns_quotactl_permission(ns, type, cmd, id);
+	if (ret < 0)
+		goto out_unlock;
+
+	if (!ns->ns_qcop) {
+		ret = -ENOSYS;
+		goto out_unlock;
+	}
+
+	switch (cmd) {
+	case Q_QUOTAON:
+		ret = ns_quota_on(ns, type);
+		break;
+	case Q_QUOTAOFF:
+		ret = ns_quota_off(ns, type);
+		break;
+	case Q_GETQUOTA:
+		ret = ns_quota_getquota(ns, type, id, addr);
+		break;
+	case Q_SETQUOTA:
+		ret = ns_quota_setquota(ns, type, id, addr);
+		break;
+	case Q_GETINFO:
+		ret = ns_quota_getinfo(ns, type, addr);
+		break;
+	case Q_SETINFO:
+		ret = ns_quota_setinfo(ns, type, addr);
+		break;
+	case Q_GETFMT:
+		ret = ns_quota_getfmt(ns, type, addr);
+		break;
+	case Q_SYNC:
+		ret = 0;
+		break;
+	default:
+		ret = -EINVAL;
+		break;
+	}
+
+out_unlock:
+	unlock_mnt_ns(ns);
+	return ret;
+}
-- 
1.7.9

[PATCH 08/12] container quota: let quotactl(2) works for container.

[jeff.liu-QHcLZuEGTsvQT0dZR+AlfA]

Modify quotactl(2) to determine for container quota control operations.

Signed-off-by: Jie Liu <jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 fs/quota/quota.c |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

diff --git a/fs/quota/quota.c b/fs/quota/quota.c
index 9a39120..40f381e 100644
--- a/fs/quota/quota.c
+++ b/fs/quota/quota.c
@@ -18,6 +18,12 @@
 #include <linux/types.h>
 #include <linux/writeback.h>
 
+/* Mount namespace disk quota required */
+#include <linux/mount.h>
+#include <linux/mnt_namespace.h>
+#include "../ns_quota.h"
+#include "../ns_quotaops.h"
+
 static int check_quotactl_permission(struct super_block *sb, int type, int cmd,
 				     qid_t id)
 {
@@ -355,6 +361,14 @@ SYSCALL_DEFINE4(quotactl, unsigned int, cmd, const char __user *, special,
 	type = cmd & SUBCMDMASK;
 
 	/*
+	 * Perform mount namespace quotactl when it is invoked from container.
+	 */
+	if (do_quotactl_for_container(special)) {
+		ret = do_container_quotactl(type, cmds, id, addr);
+		goto out;
+	}
+
+	/*
 	 * As a special case Q_SYNC can be called without a specific device.
 	 * It will iterate all superblocks that have quota enabled and call
 	 * the sync action on each of them.
-- 
1.7.9

[PATCH 09/12] container quota: add container disk quota entry to Makefile.

[jeff.liu-QHcLZuEGTsvQT0dZR+AlfA]

Make container disk quota source code could be compiled.

Signed-off-by: Jie Liu <jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 fs/Makefile |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/fs/Makefile b/fs/Makefile
index 2fb9779..bf835c5 100644
--- a/fs/Makefile
+++ b/fs/Makefile
@@ -52,6 +52,7 @@ obj-$(CONFIG_GENERIC_ACL)	+= generic_acl.o
 obj-$(CONFIG_FHANDLE)		+= fhandle.o
 
 obj-y				+= quota/
+obj-$(CONFIG_NS_QUOTA)		+= ns_dquot.o ns_quota.o
 
 obj-$(CONFIG_PROC_FS)		+= proc/
 obj-$(CONFIG_SYSFS)		+= sysfs/
-- 
1.7.9

[PATCH 10/12] container quota: bill container inodes alloc/free on ext4.

[jeff.liu-QHcLZuEGTsvQT0dZR+AlfA]

Bill up container inodes allocation and free on ext4.

Signed-off-by: Jie Liu <jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 fs/ext4/ialloc.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/fs/ext4/ialloc.c b/fs/ext4/ialloc.c
index 9f9acac..510fa41 100644
--- a/fs/ext4/ialloc.c
+++ b/fs/ext4/ialloc.c
@@ -31,6 +31,8 @@
 
 #include <trace/events/ext4.h>
 
+#include "../ns_quotaops.h"
+
 /*
  * ialloc.c contains the inodes allocation and deallocation routines
  */
@@ -233,6 +235,7 @@ void ext4_free_inode(handle_t *handle, struct inode *inode)
 	dquot_initialize(inode);
 	ext4_xattr_delete_inode(handle, inode);
 	dquot_free_inode(inode);
+	ns_dquot_free_inode(inode);
 	dquot_drop(inode);
 
 	is_directory = S_ISDIR(inode->i_mode);
@@ -861,6 +864,10 @@ got:
 	if (err)
 		goto fail_drop;
 
+	err = ns_dquot_alloc_inode(inode);
+	if (err)
+		goto fail_drop;
+
 	err = ext4_init_acl(handle, inode, dir);
 	if (err)
 		goto fail_free_drop;
@@ -902,6 +909,7 @@ really_out:
 
 fail_free_drop:
 	dquot_free_inode(inode);
+	ns_dquot_free_inode(inode);
 
 fail_drop:
 	dquot_drop(inode);
-- 
1.7.9

[PATCH 11/11] container quota: bill container disk blocks alloc/free on ext4.

[jeff.liu-QHcLZuEGTsvQT0dZR+AlfA]

Bill up container disk blocks allocation and free on ext4.

Signed-off-by: Jie Liu <jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 fs/ext4/balloc.c  |    4 ++++
 fs/ext4/mballoc.c |   17 ++++++++++++++---
 fs/ext4/xattr.c   |    7 +++++++
 3 files changed, 25 insertions(+), 3 deletions(-)

diff --git a/fs/ext4/balloc.c b/fs/ext4/balloc.c
index c45c411..417d1de 100644
--- a/fs/ext4/balloc.c
+++ b/fs/ext4/balloc.c
@@ -21,6 +21,8 @@
 #include "ext4_jbd2.h"
 #include "mballoc.h"
 
+#include "../ns_quotaops.h"
+
 #include <trace/events/ext4.h>
 
 static unsigned ext4_num_base_meta_clusters(struct super_block *sb,
@@ -547,6 +549,8 @@ ext4_fsblk_t ext4_new_meta_blocks(handle_t *handle, struct inode *inode,
 		spin_unlock(&EXT4_I(inode)->i_block_reservation_lock);
 		dquot_alloc_block_nofail(inode,
 				EXT4_C2B(EXT4_SB(inode->i_sb), ar.len));
+		ns_dquot_alloc_block_nofail(inode,
+				EXT4_C2B(EXT4_SB(inode->i_sb), ar.len));
 	}
 	return ret;
 }
diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
index 99ab428..a373a9a 100644
--- a/fs/ext4/mballoc.c
+++ b/fs/ext4/mballoc.c
@@ -27,6 +27,8 @@
 #include <linux/slab.h>
 #include <trace/events/ext4.h>
 
+#include "../ns_quotaops.h"
+
 /*
  * MUSTDO:
  *   - test ext4_ext_search_left() and ext4_ext_search_right()
@@ -4271,10 +4273,14 @@ ext4_fsblk_t ext4_mb_new_blocks(handle_t *handle,
 		if (ar->flags & EXT4_MB_USE_ROOT_BLOCKS) {
 			dquot_alloc_block_nofail(ar->inode,
 						 EXT4_C2B(sbi, ar->len));
+			ns_dquot_alloc_block_nofail(ar->inode,
+						 EXT4_C2B(sbi, ar->len));
 		} else {
 			while (ar->len &&
 				dquot_alloc_block(ar->inode,
-						  EXT4_C2B(sbi, ar->len))) {
+						  EXT4_C2B(sbi, ar->len)) &&
+				ns_dquot_alloc_block(ar->inode,
+						     EXT4_C2B(sbi, ar->len))) {
 
 				ar->flags |= EXT4_MB_HINT_NOPREALLOC;
 				ar->len--;
@@ -4353,8 +4359,11 @@ repeat:
 out:
 	if (ac)
 		kmem_cache_free(ext4_ac_cachep, ac);
-	if (inquota && ar->len < inquota)
+	if (inquota && ar->len < inquota) {
 		dquot_free_block(ar->inode, EXT4_C2B(sbi, inquota - ar->len));
+		ns_dquot_free_block(ar->inode,
+				    EXT4_C2B(sbi, inquota - ar->len));
+	}
 	if (!ar->len) {
 		if (!ext4_test_inode_state(ar->inode,
 					   EXT4_STATE_DELALLOC_RESERVED))
@@ -4673,8 +4682,10 @@ do_more:
 
 	freed += count;
 
-	if (!(flags & EXT4_FREE_BLOCKS_NO_QUOT_UPDATE))
+	if (!(flags & EXT4_FREE_BLOCKS_NO_QUOT_UPDATE)) {
 		dquot_free_block(inode, EXT4_C2B(sbi, count_clusters));
+		ns_dquot_free_block(inode, EXT4_C2B(sbi, count_clusters));
+	}
 
 	/* We dirtied the bitmap block */
 	BUFFER_TRACE(bitmap_bh, "dirtied bitmap block");
diff --git a/fs/ext4/xattr.c b/fs/ext4/xattr.c
index e88748e..1d6bc49 100644
--- a/fs/ext4/xattr.c
+++ b/fs/ext4/xattr.c
@@ -61,6 +61,8 @@
 #include "xattr.h"
 #include "acl.h"
 
+#include "../ns_quotaops.h"
+
 #define BHDR(bh) ((struct ext4_xattr_header *)((bh)->b_data))
 #define ENTRY(ptr) ((struct ext4_xattr_entry *)(ptr))
 #define BFIRST(bh) ENTRY(BHDR(bh)+1)
@@ -496,6 +498,7 @@ ext4_xattr_release_block(handle_t *handle, struct inode *inode,
 		if (IS_SYNC(inode))
 			ext4_handle_sync(handle);
 		dquot_free_block(inode, 1);
+		ns_dquot_free_block(inode, 1);
 		ea_bdebug(bh, "refcount now=%d; releasing",
 			  le32_to_cpu(BHDR(bh)->h_refcount));
 	}
@@ -787,6 +790,9 @@ inserted:
 				error = dquot_alloc_block(inode, 1);
 				if (error)
 					goto cleanup;
+				error = ns_dquot_alloc_block(inode, 1);
+				if (error)
+					goto cleanup;
 				error = ext4_journal_get_write_access(handle,
 								      new_bh);
 				if (error)
@@ -881,6 +887,7 @@ cleanup:
 
 cleanup_dquot:
 	dquot_free_block(inode, 1);
+	ns_dquot_free_block(inode, 1);
 	goto cleanup;
 
 bad_block:
-- 
1.7.9

[PATCH 12/12] container quota: init/destroy container dqinfo on mount namespace.

[jeff.liu-QHcLZuEGTsvQT0dZR+AlfA]

Initialize and destroy quota info when cloning a mount namespace or
free it up when the namespace was destroyed separately.

Signed-off-by: Jie Liu <jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 fs/namespace.c |   28 ++++++++++++++++++++++++++++
 1 files changed, 28 insertions(+), 0 deletions(-)

diff --git a/fs/namespace.c b/fs/namespace.c
index 8823ba8..b12a054 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -23,6 +23,11 @@
 #include "pnode.h"
 #include "internal.h"
 
+#ifdef CONFIG_NS_QUOTA
+#include "ns_quota.h"
+#include "ns_quotaops.h"
+#endif
+
 #define HASH_SHIFT ilog2(PAGE_SIZE / sizeof(struct list_head))
 #define HASH_SIZE (1UL << HASH_SHIFT)
 
@@ -2204,6 +2209,17 @@ static struct mnt_namespace *alloc_mnt_ns(void)
 	INIT_LIST_HEAD(&new_ns->list);
 	init_waitqueue_head(&new_ns->poll);
 	new_ns->event = 0;
+#ifdef CONFIG_NS_QUOTA
+	/*
+	 * Don't trying to enable the corresponding disk quota stuff
+	 * at the this stage for now.  We need to come up with a
+	 * reasonable definition for mount namespace or container disk
+	 * quotas.
+	 */
+	new_ns->ns_qcop = NULL;
+	new_ns->ns_dqop = NULL;
+	new_ns->ns_dqinfo = NULL;
+#endif
 	return new_ns;
 }
 
@@ -2250,6 +2266,15 @@ static struct mnt_namespace *dup_mnt_ns(struct mnt_namespace *mnt_ns,
 		return ERR_PTR(-ENOMEM);
 	}
 	new_ns->root = new;
+#ifdef CONFIG_NS_QUOTA
+	/*
+	 * Initialize disk quota stuff if the current mount namespace is
+	 * cloned for containers.
+	 */
+	new_ns->ns_qcop = &ns_quotactl_operations;
+	new_ns->ns_dqop = &ns_dquot_operations;
+	ns_dqinfo_init(new_ns);
+#endif
 	br_write_lock(vfsmount_lock);
 	list_add_tail(&new_ns->list, &new->mnt_list);
 	br_write_unlock(vfsmount_lock);
@@ -2598,6 +2623,9 @@ void put_mnt_ns(struct mnt_namespace *ns)
 	down_write(&namespace_sem);
 	br_write_lock(vfsmount_lock);
 	umount_tree(ns->root, 0, &umount_list);
+#ifdef CONFIG_NS_QUOTA
+	ns_dqinfo_destroy(ns);
+#endif
 	br_write_unlock(vfsmount_lock);
 	up_write(&namespace_sem);
 	release_mounts(&umount_list);
-- 
1.7.9

Re: [PATCH 10/12] container quota: bill container inodes alloc/free on ext4.

[Ted Ts'o]

On Wed, May 30, 2012 at 10:59:04PM +0800, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
>  	dquot_free_inode(inode);
> +	ns_dquot_free_inode(inode);

It looks like you are adding ns_dquot_<foo>() with exactly the same
arguments after dquot_<foo>() calls, for various values of <foo>.

Instead of needing to change all of the file systems, would it be
possible simply to change the quota layer to call the ns_dquot_*
functions?  It might make your life simpler, and it would certainly
reduce the number of filesystem maintainers that would need to look
through and review your changes.  You'd just have to work with Jan
Kara as the quota system maintainer.

Regards,

						- Ted

Re: [PATCH 10/12] container quota: bill container inodes alloc/free on ext4.

[Jeff Liu]

Hi Ted,

Thanks for your response!
On 05/30/2012 11:55 PM, Ted Ts'o wrote:

> On Wed, May 30, 2012 at 10:59:04PM +0800, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
>>  	dquot_free_inode(inode);
>> +	ns_dquot_free_inode(inode);
> 
> It looks like you are adding ns_dquot_<foo>() with exactly the same
> arguments after dquot_<foo>() calls, for various values of <foo>.
> 
> Instead of needing to change all of the file systems, would it be
> possible simply to change the quota layer to call the ns_dquot_*
> functions?

I also think that is stupid to change all of the file systems with those
hook routines.
At first I have tried to change the quota layer by adding
ns_dquot_<foo>() to the corresponding dquot_<foo>(), it could works on
ext4, and it should works on other file systems which are tightly bound
to VFS quota IMHO.

However, XFS has its own quota management subsystem, with looser binding
to VFS quota.  That's why am trying to export those routines over all
the file systems and CC to the maintainers, sorry for the noise.

Nevertheless, I will try to find out a straightforward way according to
your comments.

Thanks,
-Jeff

> It might make your life simpler, and it would certainly
> reduce the number of filesystem maintainers that would need to look
> through and review your changes.  You'd just have to work with Jan
> Kara as the quota system maintainer.


> 
> Regards,
> 
> 						- Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 10/12] container quota: bill container inodes alloc/free on ext4.

[Ted Ts'o]

On Thu, May 31, 2012 at 09:43:18AM +0800, Jeff Liu wrote:
> I also think that is stupid to change all of the file systems with those
> hook routines.
> At first I have tried to change the quota layer by adding
> ns_dquot_<foo>() to the corresponding dquot_<foo>(), it could works on
> ext4, and it should works on other file systems which are tightly bound
> to VFS quota IMHO.
> 
> However, XFS has its own quota management subsystem, with looser binding
> to VFS quota.  That's why am trying to export those routines over all
> the file systems and CC to the maintainers, sorry for the noise.

No worries about the noise.  I was just trying to suggest something
that might be easier for you.

As near as I can tell XFS doesn't use the fs/quota functions at all,
yes?  But if you add it into the fs/quota functions, it should get you
integration with ext2, ext3, ext4, jfs, ocfs2, and reiserfs.  Yes?

	    	       	     	   	       - Ted

Re: [PATCH 10/12] container quota: bill container inodes alloc/free on ext4.

[Jeff Liu]

On 05/31/2012 09:54 AM, Ted Ts'o wrote:

> On Thu, May 31, 2012 at 09:43:18AM +0800, Jeff Liu wrote:
>> I also think that is stupid to change all of the file systems with those
>> hook routines.
>> At first I have tried to change the quota layer by adding
>> ns_dquot_<foo>() to the corresponding dquot_<foo>(), it could works on
>> ext4, and it should works on other file systems which are tightly bound
>> to VFS quota IMHO.
>>
>> However, XFS has its own quota management subsystem, with looser binding
>> to VFS quota.  That's why am trying to export those routines over all
>> the file systems and CC to the maintainers, sorry for the noise.
> 
> No worries about the noise.  I was just trying to suggest something
> that might be easier for you.
> 
> As near as I can tell XFS doesn't use the fs/quota functions at all,
> yes?  But if you add it into the fs/quota functions, it should get you
> integration with ext2, ext3, ext4, jfs, ocfs2, and reiserfs.  Yes?

Yes, I just took a quick look through over ocfs2, reiserfs and jfs, they
are all ok.

Thanks,
-Jeff

> 
> 	    	       	     	   	       - Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 10/12] container quota: bill container inodes alloc/free on ext4.

[Jeff Liu]

On 05/31/2012 10:37 AM, Jeff Liu wrote:

> On 05/31/2012 09:54 AM, Ted Ts'o wrote:
> 
>> On Thu, May 31, 2012 at 09:43:18AM +0800, Jeff Liu wrote:
>>> I also think that is stupid to change all of the file systems with those
>>> hook routines.
>>> At first I have tried to change the quota layer by adding
>>> ns_dquot_<foo>() to the corresponding dquot_<foo>(), it could works on
>>> ext4, and it should works on other file systems which are tightly bound
>>> to VFS quota IMHO.
>>>
>>> However, XFS has its own quota management subsystem, with looser binding
>>> to VFS quota.  That's why am trying to export those routines over all
>>> the file systems and CC to the maintainers, sorry for the noise.
>>
>> No worries about the noise.  I was just trying to suggest something
>> that might be easier for you.
>>
>> As near as I can tell XFS doesn't use the fs/quota functions at all,
>> yes?  But if you add it into the fs/quota functions, it should get you
>> integration with ext2, ext3, ext4, jfs, ocfs2, and reiserfs.  Yes?
> 
> Yes, I just took a quick look through over ocfs2, reiserfs and jfs, they
> are all ok.

Such being the case, maybe I should tight ns_dquot_<foo>() to VFS quota
to reduce duplication firstly, at lease, it could works with most file
systems.  Then consider another way to let it works with other file
systems doesn't use fs/quota.

Thanks,
-Jeff

> 
> Thanks,
> -Jeff
> 
>>
>> 	    	       	     	   	       - Ted
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: container disk quota

[Glauber Costa]

On 05/30/2012 06:58 PM, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
> Hello All,
>
> According to glauber's comments regarding container disk quota, it should be binded to mount
> namespace rather than cgroup.
>
> Per my try out, it works just fine by combining with userland quota utilitly in this way.
that's great.

I'll take a look at the patches.


>
> * Modify quotactl(2) to examine if the caller is invoked inside container.
>    implemented by checking the quota device name("rootfs" for lxc guest) or current pid namespace
>    is not the initial one, then do mount namespace quotactl if required, or goto
>    the normal quotactl procedure.

I dislike the use of "lxc" name. There is nothing lxc-specific in this, 
this is namespace-specific. lxc is just one of the container solutions 
out there, so let's keep it generic.

>
> * Also, I have not handle a couple of things for now.
>    . I think the container quota should be isolated to Jan's fs/quota/ directory.
>    . There are a dozens of helper routines at general quota, e.g,
>      struct if_dqblk<->  struct fs_disk_quota converts.
>      dquot space and inodes bill up.
>      They can be refactored as shared routines to some extents.
>    . quotastats(8) is not teached to aware container for now.
>
> Changes in quota userland utility:
> * Introduce a new quota format string "lxc" to all quota control utility, to
>    let each utility know that the user want to run container quota control. e.g:
>    quotacheck -cvugm -F "lxc" /
>    quotaon -u -F "lxc" /
>    ....
>
> * Currently, I manually created the underlying device(by editing cgroup
>    device access list and running mknod /dev/sdaX x x) for the rootfs
>    inside containers to let the cache mount points routine pass for
>    executing quotacheck against the "/" directory.  Actually, it can be
>    omitted here.
>
> * Add a new quotaio_lxc.c[.h] for container quota IO, it basically same to
>    VFS quotaio logic, I just hope to isolate container stuff here.
>
> Issues:
> * How to detect quotactl(2) is launched from container in a reasonable way.

It's a system call. It is always called by a process. The process 
belongs to a namespace. What else is needed?

> * Do we need to let container quota works for cgroup combine with unshare(1)?
>    Now the patchset is mainly works for lxc guest.  IMHO, it can be used outside
>    guest if the user desired.  In this case, the quota limits can take effort
>    among different underlying file systems if they have exported quota billing
>    routines.

I still don't understand what is the business of cgroups here. If you 
are attaching it to mount namespace, you can always infer the context 
from the calling process. I still need to look at your patches, but I 
believe that dropping the "feature" of manipulating this from outside of 
the container will save you a lot of trouble.

Please note that a process can temporarily join a namespace with 
setns(). So you can have a *utility* that does it from the outer world, 
but the kernel has no business with that. As far as we're concerned, I 
believe that you should always get your context from the current 
namespace, and forbid any usage from outside.

> * The hash table list defines(hash table size)for dquot caching for each type is
>    referred to kernel/user.c, maybe its better to define an array separatly for
>    performance optimizations.  Of course, that's all depending on my current
>    implementation is on the right road. :)
>
> * Container quota statistics, should them be calculated and exposed to /proc/fs/quota?  If the underlying file system also enabled with quotas, they will be
>    mixed up, so how about add a new proc file like "ns_quota" there?
No, this should be transferred to the process-specific proc and them 
symlinked. Take a look at "/proc/self".

>
> * Memory shrinks acquired from kswap.
>    As all dquot are cached in memory, and if the user executing quotaoff, maybe
>    I need to handle quota disable but still be kept at memory.
>    Also, add another routine to disable and remove all quotas from memory to
>    save memory directly.

I didn't read your patches yet, so take it with a grain of salt here. 
But I don't understand why you make this distinction of keeping it in 
memory only.

You could keep quota files outside of the container, and then bind mount 
them to the current location in the setup-phase.

Re: [PATCH 01/12] container quota: add kernel configuration for container quota.

[Glauber Costa]

On 05/30/2012 06:58 PM, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
> Add kernel configuration for container quota.
>
> Signed-off-by: Jie Liu<jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> ---
>   fs/quota/Kconfig |    9 +++++++++
>   1 files changed, 9 insertions(+), 0 deletions(-)
>
> diff --git a/fs/quota/Kconfig b/fs/quota/Kconfig
> index 880fd98..2dd3e83 100644
> --- a/fs/quota/Kconfig
> +++ b/fs/quota/Kconfig
> @@ -16,6 +16,15 @@ config QUOTA
>   	  with the quota tools. Probably the quota support is only useful for
>   	  multi user systems. If unsure, say N.
>
> +config NS_QUOTA
> +	bool "Mount namespace quota support"
> +	depends on QUOTA
> +	default n
> +	help
> +	  If you say Y here, you will be able to set per container user limits
> +	  for disk usage (also called disk quotas). Currently, it works for the
> +	  ext4 file system only for demonstration purpose.
> +

That should depend on mount namespaces as well.

Re: [PATCH 01/12] container quota: add kernel configuration for container quota.

[Glauber Costa]

On 05/31/2012 01:00 PM, Glauber Costa wrote:
> On 05/30/2012 06:58 PM, jeff.liu@oracle.com wrote:
>> Add kernel configuration for container quota.
>>
>> Signed-off-by: Jie Liu<jeff.liu@oracle.com>
>> ---
>> fs/quota/Kconfig | 9 +++++++++
>> 1 files changed, 9 insertions(+), 0 deletions(-)
>>
>> diff --git a/fs/quota/Kconfig b/fs/quota/Kconfig
>> index 880fd98..2dd3e83 100644
>> --- a/fs/quota/Kconfig
>> +++ b/fs/quota/Kconfig
>> @@ -16,6 +16,15 @@ config QUOTA
>> with the quota tools. Probably the quota support is only useful for
>> multi user systems. If unsure, say N.
>>
>> +config NS_QUOTA
>> + bool "Mount namespace quota support"
>> + depends on QUOTA
>> + default n
>> + help
>> + If you say Y here, you will be able to set per container user limits
>> + for disk usage (also called disk quotas). Currently, it works for the
>> + ext4 file system only for demonstration purpose.
>> +
>
> That should depend on mount namespaces as well.
>

Which it already does, since there seems to be no switch for it.
duh

Re: [PATCH 02/12] container quota: lock/unlock mount namespace when performing quotactl.

[Glauber Costa]

On 05/30/2012 06:58 PM, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
> Introduce two functions to lock/unlock a particular mount namespace
> when container quota operations is performed.
>
> Signed-off-by: Jie Liu<jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> ---
>   fs/namespace.c                |   17 +++++++++++++++++
>   include/linux/mnt_namespace.h |   13 +++++++++++++
>   2 files changed, 30 insertions(+), 0 deletions(-)
>
> diff --git a/fs/namespace.c b/fs/namespace.c
> index e608199..8823ba8 100644
> --- a/fs/namespace.c
> +++ b/fs/namespace.c
> @@ -2633,3 +2633,20 @@ bool our_mnt(struct vfsmount *mnt)
>   {
>   	return check_mnt(real_mount(mnt));
>   }
> +
> +/*
> + * Lock the mount namespace when performing a particular container
> + * disk quotactl operation, especially to protect removing it.
> + */
> +void lock_mnt_ns(struct mnt_namespace *ns)
> +{
> +	down_write(&namespace_sem);
> +}
> +
> +/*
> + * Unlock the mount namespace after quotactl on that was done.
> + */
> +void unlock_mnt_ns(struct mnt_namespace *ns)
> +{
> +	up_write(&namespace_sem);
> +}
> diff --git a/include/linux/mnt_namespace.h b/include/linux/mnt_namespace.h
> index 5a8e390..ff72ed5 100644
> --- a/include/linux/mnt_namespace.h
> +++ b/include/linux/mnt_namespace.h
> @@ -9,6 +9,19 @@ extern struct mnt_namespace *copy_mnt_ns(unsigned long, struct mnt_namespace *,
>   		struct fs_struct *);
>   extern void put_mnt_ns(struct mnt_namespace *ns);
>
> +#ifdef CONFIG_NS_QUOTA
> +extern void lock_mnt_ns(struct mnt_namespace *);
> +extern void unlock_mnt_ns(struct mnt_namespace *);
> +#else
> +void lock_mnt_ns(struct mnt_namespace *)
> +{
> +}
> +
> +void unlock_mnt_ns(struct mnt_namespace *)
> +{
> +}
> +#endif
> +

Seems weird. There is nothing ns-quota specific in locking a namespace.
Also, just grabbing a lock from an outer subsystem is quite ugly. If you 
are grabbing a namespace lock, this suggests you are tweaking namespace 
data. So you should have that file provide you with a function that does 
the full operation.

Of course there are exceptions to that, but it rings a bell.

Re: [PATCH 04/12] container quota: introduce container disk quota data header file.

[Glauber Costa]

> +
> +/* Operations for quota control */
> +struct ns_quotactl_ops {
> +	int (*quota_on)(struct mnt_namespace *, int);
> +	int (*quota_off)(struct mnt_namespace *, int);
> +	int (*get_info)(struct mnt_namespace *, int, struct if_dqinfo *);
> +	int (*set_info)(struct mnt_namespace *, int, struct if_dqinfo *);
> +	int (*get_dqblk)(struct mnt_namespace *, int, qid_t,
> +			 struct fs_disk_quota *);
> +	int (*set_dqblk)(struct mnt_namespace *, int, qid_t,
> +			 struct fs_disk_quota *);
> +};
> +

That is quite bad. Just have the same signature. Which namespace you 
belong to can *always* be derived from the calling process, you should 
never need to specify that in any interface. It is of course fine to do 
that in functions internal to the kernel, but not this.

You should rethink the whole ns_quota thing. Some of it I believe will 
have to stay, I doubt we'll be able to be completely transparent. But 
the core of your changes have to be making sure a hierarchical walk over 
the valid quotas work well, finding out what are those valid quotas, etc.

The interface should be kept as unchanged as possible, and this can be 
achieved by things like automatic discover of the current namespace, and 
pre-operational container setup for the relevant files.

Re: [PATCH 10/12] container quota: bill container inodes alloc/free on ext4.

[Glauber Costa]

On 05/30/2012 06:59 PM, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
> +#include "../ns_quotaops.h"
> +
>   /*
>    * ialloc.c contains the inodes allocation and deallocation routines
>    */
> @@ -233,6 +235,7 @@ void ext4_free_inode(handle_t *handle, struct inode *inode)
>   	dquot_initialize(inode);
>   	ext4_xattr_delete_inode(handle, inode);
>   	dquot_free_inode(inode);
> +	ns_dquot_free_inode(inode);
>   	dquot_drop(inode);

This should be inside dquot_free_inode().

No need to go patch inode.c

Re: container disk quota

[Glauber Costa]

On 05/31/2012 12:54 PM, Glauber Costa wrote:
> On 05/30/2012 06:58 PM, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
>> Hello All,
>>
>> According to glauber's comments regarding container disk quota, it
>> should be binded to mount
>> namespace rather than cgroup.
>>
>> Per my try out, it works just fine by combining with userland quota
>> utilitly in this way.
> that's great.
>
> I'll take a look at the patches.

Despite my criticism, I do believe those are a lot better than the 
previous version. It seems to me to be the right direction, but some 
more transparency is lacking.

I believe you should be able to do it without a lot less changes to the 
tools - if any.

Re: container disk quota

[Jeff Liu]

Hi Glauber,

Thanks for you comments!

On 05/31/2012 04:54 PM, Glauber Costa wrote:

> On 05/30/2012 06:58 PM, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
>> Hello All,
>>
>> According to glauber's comments regarding container disk quota, it
>> should be binded to mount
>> namespace rather than cgroup.
>>
>> Per my try out, it works just fine by combining with userland quota
>> utilitly in this way.
> that's great.
> 
> I'll take a look at the patches.
> 
> 
>>
>> * Modify quotactl(2) to examine if the caller is invoked inside
>> container.
>>    implemented by checking the quota device name("rootfs" for lxc
>> guest) or current pid namespace
>>    is not the initial one, then do mount namespace quotactl if
>> required, or goto
>>    the normal quotactl procedure.
> 
> I dislike the use of "lxc" name. There is nothing lxc-specific in this,
> this is namespace-specific. lxc is just one of the container solutions
> out there, so let's keep it generic.

I think I should forget all things regarding LXC, just treat it as a new
quota feature with regard to namespace.

>>
>> * Also, I have not handle a couple of things for now.
>>    . I think the container quota should be isolated to Jan's fs/quota/
>> directory.
>>    . There are a dozens of helper routines at general quota, e.g,
>>      struct if_dqblk<->  struct fs_disk_quota converts.
>>      dquot space and inodes bill up.
>>      They can be refactored as shared routines to some extents.
>>    . quotastats(8) is not teached to aware container for now.
>>
>> Changes in quota userland utility:
>> * Introduce a new quota format string "lxc" to all quota control
>> utility, to
>>    let each utility know that the user want to run container quota
>> control. e.g:
>>    quotacheck -cvugm -F "lxc" /
>>    quotaon -u -F "lxc" /
>>    ....
>>
>> * Currently, I manually created the underlying device(by editing cgroup
>>    device access list and running mknod /dev/sdaX x x) for the rootfs
>>    inside containers to let the cache mount points routine pass for
>>    executing quotacheck against the "/" directory.  Actually, it can be
>>    omitted here.
>>
>> * Add a new quotaio_lxc.c[.h] for container quota IO, it basically
>> same to
>>    VFS quotaio logic, I just hope to isolate container stuff here.
>>
>> Issues:
>> * How to detect quotactl(2) is launched from container in a reasonable
>> way.
> 
> It's a system call. It is always called by a process. The process
> belongs to a namespace. What else is needed?

nothing now. :)

> 
>> * Do we need to let container quota works for cgroup combine with
>> unshare(1)?
>>    Now the patchset is mainly works for lxc guest.  IMHO, it can be
>> used outside
>>    guest if the user desired.  In this case, the quota limits can take
>> effort
>>    among different underlying file systems if they have exported quota
>> billing
>>    routines.
> 
> I still don't understand what is the business of cgroups here. If you
> are attaching it to mount namespace, you can always infer the context
> from the calling process. I still need to look at your patches, but I
> believe that dropping the "feature" of manipulating this from outside of
> the container will save you a lot of trouble.

Yup, just treat it to be namespace specific, there is nothing need to
consider with cgroup interface.

> 
> Please note that a process can temporarily join a namespace with
> setns(). So you can have a *utility* that does it from the outer world,
> but the kernel has no business with that. As far as we're concerned, I
> believe that you should always get your context from the current
> namespace, and forbid any usage from outside.

I'll more investigation for that.

> 
>> * The hash table list defines(hash table size)for dquot caching for
>> each type is
>>    referred to kernel/user.c, maybe its better to define an array
>> separatly for
>>    performance optimizations.  Of course, that's all depending on my
>> current
>>    implementation is on the right road. :)
>>
>> * Container quota statistics, should them be calculated and exposed to
>> /proc/fs/quota?  If the underlying file system also enabled with
>> quotas, they will be
>>    mixed up, so how about add a new proc file like "ns_quota" there?
> No, this should be transferred to the process-specific proc and them
> symlinked. Take a look at "/proc/self".
> 
>>
>> * Memory shrinks acquired from kswap.
>>    As all dquot are cached in memory, and if the user executing
>> quotaoff, maybe
>>    I need to handle quota disable but still be kept at memory.
>>    Also, add another routine to disable and remove all quotas from
>> memory to
>>    save memory directly.
> 
> I didn't read your patches yet, so take it with a grain of salt here.
> But I don't understand why you make this distinction of keeping it in
> memory only.
> 
> You could keep quota files outside of the container, and then bind mount
> them to the current location in the setup-phase.

I have tried to keep quota files outsides originally, but I changed my
thoughts afterwards, because of three reasons at that time:

1) The quota files could be overwrote if the container's rootfs is
located at the root directory of a storage partition, and this partition
is mounted with quota limits enabled.

2) To deal with quota files, looks I have to tweak up
quota_read()/quota_write(), assuming ext4, which are corresponding to
ext4_quota_read()/ext4_quota_write().

3) As mount namespace could be created and destroyed at any stage,
it has no memory to recall which inodes are quota files. however, quota
tools need to restore a few things from those files I remember.
but can not recalled all of them for now. :( I'll do some check up to
refresh my head in this point.

Sure, considering that we can bind mount them at setup phase, the first
concern could be ignored.


Thanks,
-Jeff

Re: [PATCH 02/12] container quota: lock/unlock mount namespace when performing quotactl.

[Jeff Liu]

On 05/31/2012 05:04 PM, Glauber Costa wrote:

> On 05/30/2012 06:58 PM, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
>> Introduce two functions to lock/unlock a particular mount namespace
>> when container quota operations is performed.
>>
>> Signed-off-by: Jie Liu<jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
>> ---
>>   fs/namespace.c                |   17 +++++++++++++++++
>>   include/linux/mnt_namespace.h |   13 +++++++++++++
>>   2 files changed, 30 insertions(+), 0 deletions(-)
>>
>> diff --git a/fs/namespace.c b/fs/namespace.c
>> index e608199..8823ba8 100644
>> --- a/fs/namespace.c
>> +++ b/fs/namespace.c
>> @@ -2633,3 +2633,20 @@ bool our_mnt(struct vfsmount *mnt)
>>   {
>>       return check_mnt(real_mount(mnt));
>>   }
>> +
>> +/*
>> + * Lock the mount namespace when performing a particular container
>> + * disk quotactl operation, especially to protect removing it.
>> + */
>> +void lock_mnt_ns(struct mnt_namespace *ns)
>> +{
>> +    down_write(&namespace_sem);
>> +}
>> +
>> +/*
>> + * Unlock the mount namespace after quotactl on that was done.
>> + */
>> +void unlock_mnt_ns(struct mnt_namespace *ns)
>> +{
>> +    up_write(&namespace_sem);
>> +}
>> diff --git a/include/linux/mnt_namespace.h
>> b/include/linux/mnt_namespace.h
>> index 5a8e390..ff72ed5 100644
>> --- a/include/linux/mnt_namespace.h
>> +++ b/include/linux/mnt_namespace.h
>> @@ -9,6 +9,19 @@ extern struct mnt_namespace *copy_mnt_ns(unsigned
>> long, struct mnt_namespace *,
>>           struct fs_struct *);
>>   extern void put_mnt_ns(struct mnt_namespace *ns);
>>
>> +#ifdef CONFIG_NS_QUOTA
>> +extern void lock_mnt_ns(struct mnt_namespace *);
>> +extern void unlock_mnt_ns(struct mnt_namespace *);
>> +#else
>> +void lock_mnt_ns(struct mnt_namespace *)
>> +{
>> +}
>> +
>> +void unlock_mnt_ns(struct mnt_namespace *)
>> +{
>> +}
>> +#endif
>> +
> 
> Seems weird. There is nothing ns-quota specific in locking a namespace.
> Also, just grabbing a lock from an outer subsystem is quite ugly. If you
> are grabbing a namespace lock, this suggests you are tweaking namespace
> data. So you should have that file provide you with a function that does
> the full operation.

Maybe this can be safely removed, it is presented like a place holder,
which is suggests to protect changing data inner quota info of a mount
namespace.
However, both quota info initialization and destroy business are all
projected at mount init and put mount namespace stage, so that the
lock/unlock are make less sense here.

Thanks,
-Jeff

> 
> Of course there are exceptions to that, but it rings a bell.

Re: [PATCH 04/12] container quota: introduce container disk quota data header file.

[Jeff Liu]

On 05/31/2012 05:10 PM, Glauber Costa wrote:

>> +
>> +/* Operations for quota control */
>> +struct ns_quotactl_ops {
>> +    int (*quota_on)(struct mnt_namespace *, int);
>> +    int (*quota_off)(struct mnt_namespace *, int);
>> +    int (*get_info)(struct mnt_namespace *, int, struct if_dqinfo *);
>> +    int (*set_info)(struct mnt_namespace *, int, struct if_dqinfo *);
>> +    int (*get_dqblk)(struct mnt_namespace *, int, qid_t,
>> +             struct fs_disk_quota *);
>> +    int (*set_dqblk)(struct mnt_namespace *, int, qid_t,
>> +             struct fs_disk_quota *);
>> +};
>> +
> 
> That is quite bad. Just have the same signature. Which namespace you
> belong to can *always* be derived from the calling process, you should
> never need to specify that in any interface. It is of course fine to do
> that in functions internal to the kernel, but not this.

Exactly, "struct mnt_namespace" should not be presented at any quota
operation interface, retrieve it through current->nsproxy->mnt_ns should
be fine.

> You should rethink the whole ns_quota thing. Some of it I believe will
> have to stay, I doubt we'll be able to be completely transparent. But
> the core of your changes have to be making sure a hierarchical walk over
> the valid quotas work well, finding out what are those valid quotas, etc.

> The interface should be kept as unchanged as possible, and this can be
> achieved by things like automatic discover of the current namespace, and
> pre-operational container setup for the relevant files.

Thanks for teaching me that,  let me try to refactor ns_quota to make
the change as little as possible.

-Jeff

> 
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 10/12] container quota: bill container inodes alloc/free on ext4.

[Jeff Liu]

On 05/31/2012 05:15 PM, Glauber Costa wrote:

> On 05/30/2012 06:59 PM, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
>> +#include "../ns_quotaops.h"
>> +
>>   /*
>>    * ialloc.c contains the inodes allocation and deallocation routines
>>    */
>> @@ -233,6 +235,7 @@ void ext4_free_inode(handle_t *handle, struct
>> inode *inode)
>>       dquot_initialize(inode);
>>       ext4_xattr_delete_inode(handle, inode);
>>       dquot_free_inode(inode);
>> +    ns_dquot_free_inode(inode);
>>       dquot_drop(inode);
> 
> This should be inside dquot_free_inode().

Yeah, Ted also mentioned this before, all name space dquot bill routine
could be placed to dquot_xxx() for VFS quota user.

However, AFAICS, they still have to be exported to other file systems
which have their own quota management module, like XFS.

Thanks,
-Jeff

> 
> No need to go patch inode.c
> 
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: container disk quota

[Jeff Liu]

On 05/31/2012 05:19 PM, Glauber Costa wrote:

> On 05/31/2012 12:54 PM, Glauber Costa wrote:
>> On 05/30/2012 06:58 PM, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
>>> Hello All,
>>>
>>> According to glauber's comments regarding container disk quota, it
>>> should be binded to mount
>>> namespace rather than cgroup.
>>>
>>> Per my try out, it works just fine by combining with userland quota
>>> utilitly in this way.
>> that's great.
>>
>> I'll take a look at the patches.
> 
> Despite my criticism, I do believe those are a lot better than the
> previous version. It seems to me to be the right direction, but some
> more transparency is lacking.

Thanks for your timely response and verification.

> 
> I believe you should be able to do it without a lot less changes to the
> tools - if any.

Let me try!

-Jeff

> 
> 
> 
> 
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 10/12] container quota: bill container inodes alloc/free on ext4.

[Glauber Costa]

On 05/31/2012 04:58 PM, Jeff Liu wrote:
>> >  This should be inside dquot_free_inode().
> Yeah, Ted also mentioned this before, all name space dquot bill routine
> could be placed to dquot_xxx() for VFS quota user.
>
> However, AFAICS, they still have to be exported to other file systems
> which have their own quota management module, like XFS.
>
> Thanks,
> -Jeff
>
Do you have a personal need to have it working with xfs, or is the use 
you are pursuing achievable with the other filesystems?

Unless you have a reason yourself to use xfs, I'd say don't bother with 
it too much at the moment. get it working for the standard interface first.

Re: [PATCH 10/12] container quota: bill container inodes alloc/free on ext4.

[Jeff Liu]

On 05/31/2012 09:14 PM, Glauber Costa wrote:

> On 05/31/2012 04:58 PM, Jeff Liu wrote:
>>> >  This should be inside dquot_free_inode().
>> Yeah, Ted also mentioned this before, all name space dquot bill routine
>> could be placed to dquot_xxx() for VFS quota user.
>>
>> However, AFAICS, they still have to be exported to other file systems
>> which have their own quota management module, like XFS.
>>
>> Thanks,
>> -Jeff
>>
> Do you have a personal need to have it working with xfs, or is the use
> you are pursuing achievable with the other filesystems?

I am just trying to figure out a unique interface for all file systems.

> 
> Unless you have a reason yourself to use xfs, I'd say don't bother with
> it too much at the moment. get it working for the standard interface first.

Ok, It can also make my life easier.

Thanks,
-Jeff

> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: container disk quota

[Jan Kara]

  Hello,

On Wed 30-05-12 22:58:54, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
> According to glauber's comments regarding container disk quota, it should be binded to mount
> namespace rather than cgroup.
> 
> Per my try out, it works just fine by combining with userland quota utilitly in this way.
> However, they are something has to be done at user tools too IMHO.
> 
> Currently, the patchset is in very initial phase, I'd like to post it early to seek more
> feedbacks from you guys.
> 
> Hopefully I can clarify my ideas clearly.
  So what I miss in this introductory email is some highlevel description
like what is the desired functionality you try to implement and what is it
good for. Looking at the examples below, it seems you want to be able to
set quota limits for namespace-uid (and also namespace-gid???) pairs, am I
right?

  If yes, then I would like to understand one thing: When writing to a
file, used space is accounted to the owner of the file. Now how do we
determine owning namespace? Do you implicitely assume that only processes
from one namespace will be able to access the file?

								Honza
-- 
Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
SUSE Labs, CR

Re: container disk quota

[Serge Hallyn]

Quoting Jan Kara (jack-AlSwsSmVLrQ@public.gmane.org):
>   Hello,
> 
> On Wed 30-05-12 22:58:54, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
> > According to glauber's comments regarding container disk quota, it should be binded to mount
> > namespace rather than cgroup.
> > 
> > Per my try out, it works just fine by combining with userland quota utilitly in this way.
> > However, they are something has to be done at user tools too IMHO.
> > 
> > Currently, the patchset is in very initial phase, I'd like to post it early to seek more
> > feedbacks from you guys.
> > 
> > Hopefully I can clarify my ideas clearly.
>   So what I miss in this introductory email is some highlevel description
> like what is the desired functionality you try to implement and what is it
> good for. Looking at the examples below, it seems you want to be able to
> set quota limits for namespace-uid (and also namespace-gid???) pairs, am I
> right?
> 
>   If yes, then I would like to understand one thing: When writing to a
> file, used space is accounted to the owner of the file. Now how do we
> determine owning namespace? Do you implicitely assume that only processes
> from one namespace will be able to access the file?
> 
> 								Honza

Not having looked closely at the original patchset, let me ask - is this
feature going to be a freebie with Eric's usernamespace patches?

There, a container can be started in its own user namespace.  It's uid
1000 will be mapped to something like 1101000 on the host.  So the actual
uid against who the quota is counted is 1101000.  In another container,
uid 1000 will be mapped to 1201000, and again quota will be counted against
1201000.

Note that this won't work with bind mounts, as a file can only be owned
by one uid, be it 1000, 1101000, or 1201000.  So for the quota to work
each container would need its own files.  (Of course the underlying
metadata can be shared through whatever ways - btrfs, lvm snapshotting,
etc)

-serge

Re: container disk quota

[Jeff Liu]

Hi Jan,

On 06/01/2012 11:54 PM, Jan Kara wrote:

>   Hello,
> 
> On Wed 30-05-12 22:58:54, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
>> According to glauber's comments regarding container disk quota, it should be binded to mount
>> namespace rather than cgroup.
>>
>> Per my try out, it works just fine by combining with userland quota utilitly in this way.
>> However, they are something has to be done at user tools too IMHO.
>>
>> Currently, the patchset is in very initial phase, I'd like to post it early to seek more
>> feedbacks from you guys.
>>
>> Hopefully I can clarify my ideas clearly.
>   So what I miss in this introductory email is some highlevel description
> like what is the desired functionality you try to implement and what is it
> good for. Looking at the examples below, it seems you want to be able to
> set quota limits for namespace-uid (and also namespace-gid???) pairs, am I
> right?

Sorry for lacking the high level descriptions.

The main idea is to introduce a quota mechanism to let container make use of it like VFS quota/quota
tools as transparent as possible.

However our general quota subsystem works with one global hashtable list, and it based on superblock
and global UID/PID for those accounting, and looks that's not convenient to support container quota
without much changes IMHO.

The current opinion is to bind it to mount namespace, since container can be setup through CLONE_MNTNS to
have its private mount tree.

> 
>   If yes, then I would like to understand one thing: When writing to a
> file, used space is accounted to the owner of the file. Now how do we
> determine owning namespace?

That's also the main point I am concerned.
It think the latest USER_NS feature would be got involved in.
And also, we can detect the mount namespace by inferring it from processes context.  We only bill those
space operations if themount namespace is quota desired.

Here has another issue, when we doing quota info initialization on a mount namespace?
In my current implementation, it is simply implemented by checking if mount namespace was cloned out,
and do quotactl(2) if the input device is "rootfs",  or just checking if the mount namespace is not the
initial global one.
quotactl(2) will be processed if meet either above conditions, and space operations will be counted if
the desired UID/GID dquot could be searched.

> Do you implicitely assume that only processes
> from one namespace will be able to access the file?

All processes could access the file, but only those space operation of processes resides in the mount
namespace with quota setup will be billed.


Thanks,
-Jeff

> 								Honza

Re: container disk quota

[Jeff Liu]

Hi Serge,

On 06/02/2012 12:04 AM, Serge Hallyn wrote:

> Quoting Jan Kara (jack-AlSwsSmVLrQ@public.gmane.org):
>>   Hello,
>>
>> On Wed 30-05-12 22:58:54, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
>>> According to glauber's comments regarding container disk quota, it should be binded to mount
>>> namespace rather than cgroup.
>>>
>>> Per my try out, it works just fine by combining with userland quota utilitly in this way.
>>> However, they are something has to be done at user tools too IMHO.
>>>
>>> Currently, the patchset is in very initial phase, I'd like to post it early to seek more
>>> feedbacks from you guys.
>>>
>>> Hopefully I can clarify my ideas clearly.
>>   So what I miss in this introductory email is some highlevel description
>> like what is the desired functionality you try to implement and what is it
>> good for. Looking at the examples below, it seems you want to be able to
>> set quota limits for namespace-uid (and also namespace-gid???) pairs, am I
>> right?
>>
>>   If yes, then I would like to understand one thing: When writing to a
>> file, used space is accounted to the owner of the file. Now how do we
>> determine owning namespace? Do you implicitely assume that only processes
>> from one namespace will be able to access the file?
>>
>> 								Honza
> 
> Not having looked closely at the original patchset, let me ask - is this
> feature going to be a freebie with Eric's usernamespace patches?

It we can reach a consensus to bind quota on mount namespace for
container or other things maybe.
I think it definitely should depends on user namespace.

> 
> There, a container can be started in its own user namespace.  It's uid
> 1000 will be mapped to something like 1101000 on the host.  So the actual
> uid against who the quota is counted is 1101000.  In another container,
> uid 1000 will be mapped to 1201000, and again quota will be counted against
> 1201000.

Is it also an implications that we can examine do container quota or not
based on the uid/gid number?

> 
> Note that this won't work with bind mounts, as a file can only be owned
> by one uid, be it 1000, 1101000, or 1201000.  So for the quota to work
> each container would need its own files.  (Of course the underlying
> metadata can be shared through whatever ways - btrfs, lvm snapshotting,
> etc)

Do you means that we can not bind mount outside files to container for
as general adquot.user/adquot.group purpose?

If so, per glauber's comments, bind quota to mount namespace should be a
generic feature, and container just one of users could make use of it.

Again, if bind quota to mount namespace is on right direction, and it
only does make sense to container for now, maybe we don't need such
files. IMHO, container is a lightweight virtualization solution, maybe
its fine to make it as simple as possible.  If the server admin need to
configure hundreds of user/group dquot per container, perhaps he should
consider KVM/XEN.


Thanks,
-Jeff



> 
> -serge
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: container disk quota

[Kirill Korotaev]

On Jun 2, 2012, at 09:59 , Jeff Liu wrote:

> Hi Serge,
> 
> On 06/02/2012 12:04 AM, Serge Hallyn wrote:
> 
>> Quoting Jan Kara (jack@suse.cz):
>>>  Hello,
>>> 
>>> On Wed 30-05-12 22:58:54, jeff.liu@oracle.com wrote:
>>>> According to glauber's comments regarding container disk quota, it should be binded to mount
>>>> namespace rather than cgroup.
>>>> 
>>>> Per my try out, it works just fine by combining with userland quota utilitly in this way.
>>>> However, they are something has to be done at user tools too IMHO.
>>>> 
>>>> Currently, the patchset is in very initial phase, I'd like to post it early to seek more
>>>> feedbacks from you guys.
>>>> 
>>>> Hopefully I can clarify my ideas clearly.
>>>  So what I miss in this introductory email is some highlevel description
>>> like what is the desired functionality you try to implement and what is it
>>> good for. Looking at the examples below, it seems you want to be able to
>>> set quota limits for namespace-uid (and also namespace-gid???) pairs, am I
>>> right?
>>> 
>>>  If yes, then I would like to understand one thing: When writing to a
>>> file, used space is accounted to the owner of the file. Now how do we
>>> determine owning namespace? Do you implicitely assume that only processes
>>> from one namespace will be able to access the file?
>>> 
>>> 								Honza
>> 
>> Not having looked closely at the original patchset, let me ask - is this
>> feature going to be a freebie with Eric's usernamespace patches?
> 
> It we can reach a consensus to bind quota on mount namespace for
> container or other things maybe.

1. OpenVZ doesn't use mount namespaces and still has quotas per container.

2. BTW, have you seen Dmitry Monakhov patches for same containers quotas via additional inode attribute? it allows to make it journaled.
How quotas are stored in your case?

3. I tend to think nowdays such quotas maybe of less need. Quota code doesn't scale well. And it's easier to put container in image file (as OpenVZ recently introduced).

Thanks,
Kirill

Re: container disk quota

[Jeff Liu]

On 06/02/2012 02:06 PM, Kirill Korotaev wrote:

> 
> On Jun 2, 2012, at 09:59 , Jeff Liu wrote:
> 
>> Hi Serge,
>>
>> On 06/02/2012 12:04 AM, Serge Hallyn wrote:
>>
>>> Quoting Jan Kara (jack-AlSwsSmVLrQ@public.gmane.org):
>>>>  Hello,
>>>>
>>>> On Wed 30-05-12 22:58:54, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
>>>>> According to glauber's comments regarding container disk quota, it should be binded to mount
>>>>> namespace rather than cgroup.
>>>>>
>>>>> Per my try out, it works just fine by combining with userland quota utilitly in this way.
>>>>> However, they are something has to be done at user tools too IMHO.
>>>>>
>>>>> Currently, the patchset is in very initial phase, I'd like to post it early to seek more
>>>>> feedbacks from you guys.
>>>>>
>>>>> Hopefully I can clarify my ideas clearly.
>>>>  So what I miss in this introductory email is some highlevel description
>>>> like what is the desired functionality you try to implement and what is it
>>>> good for. Looking at the examples below, it seems you want to be able to
>>>> set quota limits for namespace-uid (and also namespace-gid???) pairs, am I
>>>> right?
>>>>
>>>>  If yes, then I would like to understand one thing: When writing to a
>>>> file, used space is accounted to the owner of the file. Now how do we
>>>> determine owning namespace? Do you implicitely assume that only processes
>>>> from one namespace will be able to access the file?
>>>>
>>>> 								Honza
>>>
>>> Not having looked closely at the original patchset, let me ask - is this
>>> feature going to be a freebie with Eric's usernamespace patches?
>>
>> It we can reach a consensus to bind quota on mount namespace for
>> container or other things maybe.
> 
> 1. OpenVZ doesn't use mount namespaces and still has quotas per container.

AFAICS, OpenVZ has self-released quota tools to supply this feature.

> 
> 2. BTW, have you seen Dmitry Monakhov patches for same containers quotas via additional inode attribute? it allows to make it journaled.

You means the directly/project quota on ext4?
If yes, I have observed this feature back to the end of last year in
EXT4 mail list.

> How quotas are stored in your case?

It simply cached at memory for now, it also can be tweak up to journaled
I think, if introducing corresponding routines quota_read/quota_write to
particular journal file system.

> 
> 3. I tend to think nowdays such quotas maybe of less need. Quota code doesn't scale well. And it's easier to put container in image file (as OpenVZ recently introduced).

There have such requirements dropped to LXC mail list nowadays.
Directory quota is pretty cool and it also useful to containers perspective.

However, that's two different quota mechanism.

"Quota code doesn't scale well".
Do you means it have global locking mechanism and only quota structure
to bill up quota for all file systems with VFS quota enabled?

I noticed that OpenVZ has introduced an image file to supply container
quota, and especially for the container migration consideration.
However, could it be a general solution to LXC?


Thanks,
-Jeff

> 
> Thanks,
> Kirill
> 

Re: container disk quota

[Kirill Korotaev]

>>>> 
>>>> Not having looked closely at the original patchset, let me ask - is this
>>>> feature going to be a freebie with Eric's usernamespace patches?
>>> 
>>> It we can reach a consensus to bind quota on mount namespace for
>>> container or other things maybe.
>> 
>> 1. OpenVZ doesn't use mount namespaces and still has quotas per container.
> 
> AFAICS, OpenVZ has self-released quota tools to supply this feature.

but standard quota tools work inside container w/o any modifications.
This is very important for us, cause we run unmodified distros inside.

Actually, this is unrelated. I meant that OpenVZ needs ability to have group quotas w/o mount namespaces.

> 
>> 
>> 2. BTW, have you seen Dmitry Monakhov patches for same containers quotas via additional inode attribute? it allows to make it journaled.
> 
> You means the directly/project quota on ext4?
> If yes, I have observed this feature back to the end of last year in
> EXT4 mail list.

yes

> 
>> How quotas are stored in your case?
> 
> It simply cached at memory for now, it also can be tweak up to journaled
> I think, if introducing corresponding routines quota_read/quota_write to
> particular journal file system.

just cached quotas are bad - you never sure they are correct.
journaled quotas (as standart) are much better.

> 
>> 
>> 3. I tend to think nowdays such quotas maybe of less need. Quota code doesn't scale well. And it's easier to put container in image file (as OpenVZ recently introduced).
> 
> There have such requirements dropped to LXC mail list nowadays.
> Directory quota is pretty cool and it also useful to containers perspective.
> 
> However, that's two different quota mechanism.
> 
> "Quota code doesn't scale well".
> Do you means it have global locking mechanism and only quota structure
> to bill up quota for all file systems with VFS quota enabled?

yes.

Kirill

Re: container disk quota

[Jeff Liu]

Hi Kirill,

On 06/02/2012 11:21 PM, Kirill Korotaev wrote:

>>>>>
>>>>> Not having looked closely at the original patchset, let me ask - is this
>>>>> feature going to be a freebie with Eric's usernamespace patches?
>>>>
>>>> It we can reach a consensus to bind quota on mount namespace for
>>>> container or other things maybe.
>>>
>>> 1. OpenVZ doesn't use mount namespaces and still has quotas per container.
>>
>> AFAICS, OpenVZ has self-released quota tools to supply this feature.
> 
> but standard quota tools work inside container w/o any modifications.
> This is very important for us, cause we run unmodified distros inside.

Yes, am agree.
I can work out a new patches regarding quota tools based on mount namespace w/o any modification.

> 
> Actually, this is unrelated. I meant that OpenVZ needs ability to have group quotas w/o mount namespaces.
> 
>>
>>>
>>> 2. BTW, have you seen Dmitry Monakhov patches for same containers quotas via additional inode attribute? it allows to make it journaled.
>>
>> You means the directly/project quota on ext4?
>> If yes, I have observed this feature back to the end of last year in
>> EXT4 mail list.
> 
> yes
> 
>>
>>> How quotas are stored in your case?
>>
>> It simply cached at memory for now, it also can be tweak up to journaled
>> I think, if introducing corresponding routines quota_read/quota_write to
>> particular journal file system.
> 
> just cached quotas are bad - you never sure they are correct.
> journaled quotas (as standart) are much better.

Exactly.

> 
>>
>>>
>>> 3. I tend to think nowdays such quotas maybe of less need. Quota code doesn't scale well. And it's easier to put container in image file (as OpenVZ recently introduced).
>>
>> There have such requirements dropped to LXC mail list nowadays.
>> Directory quota is pretty cool and it also useful to containers perspective.
>>
>> However, that's two different quota mechanism.
>>
>> "Quota code doesn't scale well".
>> Do you means it have global locking mechanism and only quota structure
>> to bill up quota for all file systems with VFS quota enabled?
> 
> yes.

That's also means there has a potential opportunity for improvement in terms of scalability.

Thanks for your info!
-Jeff

> 
> Kirill
> 

Re: container disk quota

[Kirill Korotaev]

On Jun 3, 2012, at 08:23 , Jeff Liu wrote:

> Hi Kirill,
> 
> On 06/02/2012 11:21 PM, Kirill Korotaev wrote:
> 
>>>>>> 
>>>>>> Not having looked closely at the original patchset, let me ask - is this
>>>>>> feature going to be a freebie with Eric's usernamespace patches?
>>>>> 
>>>>> It we can reach a consensus to bind quota on mount namespace for
>>>>> container or other things maybe.
>>>> 
>>>> 1. OpenVZ doesn't use mount namespaces and still has quotas per container.
>>> 
>>> AFAICS, OpenVZ has self-released quota tools to supply this feature.
>> 
>> but standard quota tools work inside container w/o any modifications.
>> This is very important for us, cause we run unmodified distros inside.
> 
> Yes, am agree.
> I can work out a new patches regarding quota tools based on mount namespace w/o any modification.

Jeff, why do you need fs namespace for quotas? OpenVZ works w/o it.
It sounds as too strict use case limitation. Or do I understand your patchset description wrong?

Thanks,
Kirill

Re: container disk quota

[Jeff Liu]

On 06/03/2012 01:47 PM, Kirill Korotaev wrote:

> 
> On Jun 3, 2012, at 08:23 , Jeff Liu wrote:
> 
>> Hi Kirill,
>>
>> On 06/02/2012 11:21 PM, Kirill Korotaev wrote:
>>
>>>>>>>
>>>>>>> Not having looked closely at the original patchset, let me ask - is this
>>>>>>> feature going to be a freebie with Eric's usernamespace patches?
>>>>>>
>>>>>> It we can reach a consensus to bind quota on mount namespace for
>>>>>> container or other things maybe.
>>>>>
>>>>> 1. OpenVZ doesn't use mount namespaces and still has quotas per container.
>>>>
>>>> AFAICS, OpenVZ has self-released quota tools to supply this feature.
>>>
>>> but standard quota tools work inside container w/o any modifications.
>>> This is very important for us, cause we run unmodified distros inside.
>>
>> Yes, am agree.
>> I can work out a new patches regarding quota tools based on mount namespace w/o any modification.
> 
> Jeff, why do you need fs namespace for quotas? OpenVZ works w/o it.

This idea is come from by Glauber's comments for my previous patches
which bind quota to cgroup, and per my try out, it really works to some
extents.

> It sounds as too strict use case limitation. Or do I understand your patchset description wrong?

According to my understood, bind quota to mount namespace is another
quota strategy, and container is a potential user case, maybe could be
used for other things.

I certainly will study openVZ's quota code.

Thanks,
-Jeff

> 
> Thanks,
> Kirill
> 

Re: container disk quota

[Glauber Costa]

On 06/03/2012 09:47 AM, Kirill Korotaev wrote:
>
> On Jun 3, 2012, at 08:23 , Jeff Liu wrote:
>
>> Hi Kirill,
>>
>> On 06/02/2012 11:21 PM, Kirill Korotaev wrote:
>>
>>>>>>>
>>>>>>> Not having looked closely at the original patchset, let me ask - is this
>>>>>>> feature going to be a freebie with Eric's usernamespace patches?
>>>>>>
>>>>>> It we can reach a consensus to bind quota on mount namespace for
>>>>>> container or other things maybe.
>>>>>
>>>>> 1. OpenVZ doesn't use mount namespaces and still has quotas per container.
>>>>
>>>> AFAICS, OpenVZ has self-released quota tools to supply this feature.
>>>
>>> but standard quota tools work inside container w/o any modifications.
>>> This is very important for us, cause we run unmodified distros inside.
>>
>> Yes, am agree.
>> I can work out a new patches regarding quota tools based on mount namespace w/o any modification.
>
> Jeff, why do you need fs namespace for quotas? OpenVZ works w/o it.
> It sounds as too strict use case limitation. Or do I understand your patchset description wrong?
>

OpenVZ has its own kernel, with its very own isolation capabilities.
So in upstream, something has to be used instead.

Re: container disk quota

[Serge Hallyn]

Quoting Jeff Liu (jeff.liu@oracle.com):
> Hi Serge,
> 
> On 06/02/2012 12:04 AM, Serge Hallyn wrote:
> 
> > Quoting Jan Kara (jack@suse.cz):
> >>   Hello,
> >>
> >> On Wed 30-05-12 22:58:54, jeff.liu@oracle.com wrote:
> >>> According to glauber's comments regarding container disk quota, it should be binded to mount
> >>> namespace rather than cgroup.
> >>>
> >>> Per my try out, it works just fine by combining with userland quota utilitly in this way.
> >>> However, they are something has to be done at user tools too IMHO.
> >>>
> >>> Currently, the patchset is in very initial phase, I'd like to post it early to seek more
> >>> feedbacks from you guys.
> >>>
> >>> Hopefully I can clarify my ideas clearly.
> >>   So what I miss in this introductory email is some highlevel description
> >> like what is the desired functionality you try to implement and what is it
> >> good for. Looking at the examples below, it seems you want to be able to
> >> set quota limits for namespace-uid (and also namespace-gid???) pairs, am I
> >> right?
> >>
> >>   If yes, then I would like to understand one thing: When writing to a
> >> file, used space is accounted to the owner of the file. Now how do we
> >> determine owning namespace? Do you implicitely assume that only processes
> >> from one namespace will be able to access the file?
> >>
> >> 								Honza
> > 
> > Not having looked closely at the original patchset, let me ask - is this
> > feature going to be a freebie with Eric's usernamespace patches?
> 
> It we can reach a consensus to bind quota on mount namespace for
> container or other things maybe.
> I think it definitely should depends on user namespace.
> 
> > 
> > There, a container can be started in its own user namespace.  It's uid
> > 1000 will be mapped to something like 1101000 on the host.  So the actual
> > uid against who the quota is counted is 1101000.  In another container,
> > uid 1000 will be mapped to 1201000, and again quota will be counted against
> > 1201000.
> 
> Is it also an implications that we can examine do container quota or not
> based on the uid/gid number?

I'm sorry I don't understand the question.

As an attempt at an answer:  the quota code wouldn't change at all.  We would
simply exploit the fact that uid 1000 in container1 has a real uid of 101100,
which is different from the real uid 102100 assigned to uid 1000 in container2
and from real uid 1000 (uid 1000 on the host).

> > Note that this won't work with bind mounts, as a file can only be owned
> > by one uid, be it 1000, 1101000, or 1201000.  So for the quota to work
> > each container would need its own files.  (Of course the underlying
> > metadata can be shared through whatever ways - btrfs, lvm snapshotting,
> > etc)
> 
> Do you means that we can not bind mount outside files to container for
> as general adquot.user/adquot.group purpose?

Right, not without some sort of stackable filesystem which masks the uid.

Actually there may be a way around it (simply provide a mount option,
requiring privilege in the original user namespace, saying mask uid x to
look like uid y for this bind mount), but it's too early to say how
cleanly that could be done.

> If so, per glauber's comments, bind quota to mount namespace should be a
> generic feature, and container just one of users could make use of it.
> 
> Again, if bind quota to mount namespace is on right direction, and it
> only does make sense to container for now, maybe we don't need such
> files. IMHO, container is a lightweight virtualization solution, maybe
> its fine to make it as simple as possible.  If the server admin need to
> configure hundreds of user/group dquot per container, perhaps he should
> consider KVM/XEN.

Server admin doesn't need to do that.

-serge

Re: container disk quota

[Jeff Liu]

On 06/04/2012 10:57 AM, Serge Hallyn wrote:

> Quoting Jeff Liu (jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org):
>> Hi Serge,
>>
>> On 06/02/2012 12:04 AM, Serge Hallyn wrote:
>>
>>> Quoting Jan Kara (jack-AlSwsSmVLrQ@public.gmane.org):
>>>>   Hello,
>>>>
>>>> On Wed 30-05-12 22:58:54, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
>>>>> According to glauber's comments regarding container disk quota, it should be binded to mount
>>>>> namespace rather than cgroup.
>>>>>
>>>>> Per my try out, it works just fine by combining with userland quota utilitly in this way.
>>>>> However, they are something has to be done at user tools too IMHO.
>>>>>
>>>>> Currently, the patchset is in very initial phase, I'd like to post it early to seek more
>>>>> feedbacks from you guys.
>>>>>
>>>>> Hopefully I can clarify my ideas clearly.
>>>>   So what I miss in this introductory email is some highlevel description
>>>> like what is the desired functionality you try to implement and what is it
>>>> good for. Looking at the examples below, it seems you want to be able to
>>>> set quota limits for namespace-uid (and also namespace-gid???) pairs, am I
>>>> right?
>>>>
>>>>   If yes, then I would like to understand one thing: When writing to a
>>>> file, used space is accounted to the owner of the file. Now how do we
>>>> determine owning namespace? Do you implicitely assume that only processes
>>>> from one namespace will be able to access the file?
>>>>
>>>> 								Honza
>>>
>>> Not having looked closely at the original patchset, let me ask - is this
>>> feature going to be a freebie with Eric's usernamespace patches?
>>
>> It we can reach a consensus to bind quota on mount namespace for
>> container or other things maybe.
>> I think it definitely should depends on user namespace.
>>
>>>
>>> There, a container can be started in its own user namespace.  It's uid
>>> 1000 will be mapped to something like 1101000 on the host.  So the actual
>>> uid against who the quota is counted is 1101000.  In another container,
>>> uid 1000 will be mapped to 1201000, and again quota will be counted against
>>> 1201000.
>>
>> Is it also an implications that we can examine do container quota or not
>> based on the uid/gid number?
> 
> I'm sorry I don't understand the question.

Sorry for my poor english.

> 
> As an attempt at an answer:  the quota code wouldn't change at all.  We would
> simply exploit the fact that uid 1000 in container1 has a real uid of 101100,
> which is different from the real uid 102100 assigned to uid 1000 in container2
> and from real uid 1000 (uid 1000 on the host).

In that case, looks we only need to figure out how to let quota tools
works at container.
I'll build a new kernel with user_ns to give a try.

> 
>>> Note that this won't work with bind mounts, as a file can only be owned
>>> by one uid, be it 1000, 1101000, or 1201000.  So for the quota to work
>>> each container would need its own files.  (Of course the underlying
>>> metadata can be shared through whatever ways - btrfs, lvm snapshotting,
>>> etc)
>>
>> Do you means that we can not bind mount outside files to container for
>> as general adquot.user/adquot.group purpose?
> 
> Right, not without some sort of stackable filesystem which masks the uid.
> 
> Actually there may be a way around it (simply provide a mount option,
> requiring privilege in the original user namespace, saying mask uid x to
> look like uid y for this bind mount), but it's too early to say how
> cleanly that could be done.

> 
>> If so, per glauber's comments, bind quota to mount namespace should be a
>> generic feature, and container just one of users could make use of it.
>>
>> Again, if bind quota to mount namespace is on right direction, and it
>> only does make sense to container for now, maybe we don't need such
>> files. IMHO, container is a lightweight virtualization solution, maybe
>> its fine to make it as simple as possible.  If the server admin need to
>> configure hundreds of user/group dquot per container, perhaps he should
>> consider KVM/XEN.
> 
> Server admin doesn't need to do that.

Thanks for the info!

-Jeff

> 
> -serge
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: container disk quota

[Jan Kara]

On Mon 04-06-12 12:46:49, Jeff Liu wrote:
> On 06/04/2012 10:57 AM, Serge Hallyn wrote:
> > Quoting Jeff Liu (jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org):
> >> Hi Serge,
> >>
> >> On 06/02/2012 12:04 AM, Serge Hallyn wrote:
> >>
> >>> Quoting Jan Kara (jack-AlSwsSmVLrQ@public.gmane.org):
> >>>>   Hello,
> >>>>
> >>>> On Wed 30-05-12 22:58:54, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
> >>>>> According to glauber's comments regarding container disk quota, it should be binded to mount
> >>>>> namespace rather than cgroup.
> >>>>>
> >>>>> Per my try out, it works just fine by combining with userland quota utilitly in this way.
> >>>>> However, they are something has to be done at user tools too IMHO.
> >>>>>
> >>>>> Currently, the patchset is in very initial phase, I'd like to post it early to seek more
> >>>>> feedbacks from you guys.
> >>>>>
> >>>>> Hopefully I can clarify my ideas clearly.
> >>>>   So what I miss in this introductory email is some highlevel description
> >>>> like what is the desired functionality you try to implement and what is it
> >>>> good for. Looking at the examples below, it seems you want to be able to
> >>>> set quota limits for namespace-uid (and also namespace-gid???) pairs, am I
> >>>> right?
> >>>>
> >>>>   If yes, then I would like to understand one thing: When writing to a
> >>>> file, used space is accounted to the owner of the file. Now how do we
> >>>> determine owning namespace? Do you implicitely assume that only processes
> >>>> from one namespace will be able to access the file?
> >>>>
> >>>> 								Honza
> >>>
> >>> Not having looked closely at the original patchset, let me ask - is this
> >>> feature going to be a freebie with Eric's usernamespace patches?
> >>
> >> It we can reach a consensus to bind quota on mount namespace for
> >> container or other things maybe.
> >> I think it definitely should depends on user namespace.
> >>
> >>>
> >>> There, a container can be started in its own user namespace.  It's uid
> >>> 1000 will be mapped to something like 1101000 on the host.  So the actual
> >>> uid against who the quota is counted is 1101000.  In another container,
> >>> uid 1000 will be mapped to 1201000, and again quota will be counted against
> >>> 1201000.
> >>
> >> Is it also an implications that we can examine do container quota or not
> >> based on the uid/gid number?
> > 
> > I'm sorry I don't understand the question.
> 
> Sorry for my poor english.
> 
> > 
> > As an attempt at an answer:  the quota code wouldn't change at all.  We would
> > simply exploit the fact that uid 1000 in container1 has a real uid of 101100,
> > which is different from the real uid 102100 assigned to uid 1000 in container2
> > and from real uid 1000 (uid 1000 on the host).
> 
> In that case, looks we only need to figure out how to let quota tools
> works at container.
> I'll build a new kernel with user_ns to give a try.
  GETQUOTA or SETQUOTA quotactls should work just fine inside a container
(for those quota-tools just need access to /proc/mounts). QUOTAON should
also work for e.g. XFS or ext4 with hidden quota files. When quota files
are visible in fs namespace (as for ext3 or so), things would be a bit
tricky because they won't be possibly visible from container and QUOTAON
needs that.

Also with QUOTAON there is the principial problem that quotas either are or
are not enabled for the whole filesystem. So probably the only reasonable
choice when you would like to supporot quotas in the container would be to
have quotas enabled all the time, and inside the container, you would just
set some quota limits or you won't...

								Honza
-- 
Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
SUSE Labs, CR

Re: container disk quota

[Jeff Liu]

On 06/04/2012 05:42 PM, Jan Kara wrote:

> On Mon 04-06-12 12:46:49, Jeff Liu wrote:
>> On 06/04/2012 10:57 AM, Serge Hallyn wrote:
>>> Quoting Jeff Liu (jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org):
>>>> Hi Serge,
>>>>
>>>> On 06/02/2012 12:04 AM, Serge Hallyn wrote:
>>>>
>>>>> Quoting Jan Kara (jack-AlSwsSmVLrQ@public.gmane.org):
>>>>>>   Hello,
>>>>>>
>>>>>> On Wed 30-05-12 22:58:54, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
>>>>>>> According to glauber's comments regarding container disk quota, it should be binded to mount
>>>>>>> namespace rather than cgroup.
>>>>>>>
>>>>>>> Per my try out, it works just fine by combining with userland quota utilitly in this way.
>>>>>>> However, they are something has to be done at user tools too IMHO.
>>>>>>>
>>>>>>> Currently, the patchset is in very initial phase, I'd like to post it early to seek more
>>>>>>> feedbacks from you guys.
>>>>>>>
>>>>>>> Hopefully I can clarify my ideas clearly.
>>>>>>   So what I miss in this introductory email is some highlevel description
>>>>>> like what is the desired functionality you try to implement and what is it
>>>>>> good for. Looking at the examples below, it seems you want to be able to
>>>>>> set quota limits for namespace-uid (and also namespace-gid???) pairs, am I
>>>>>> right?
>>>>>>
>>>>>>   If yes, then I would like to understand one thing: When writing to a
>>>>>> file, used space is accounted to the owner of the file. Now how do we
>>>>>> determine owning namespace? Do you implicitely assume that only processes
>>>>>> from one namespace will be able to access the file?
>>>>>>
>>>>>> 								Honza
>>>>>
>>>>> Not having looked closely at the original patchset, let me ask - is this
>>>>> feature going to be a freebie with Eric's usernamespace patches?
>>>>
>>>> It we can reach a consensus to bind quota on mount namespace for
>>>> container or other things maybe.
>>>> I think it definitely should depends on user namespace.
>>>>
>>>>>
>>>>> There, a container can be started in its own user namespace.  It's uid
>>>>> 1000 will be mapped to something like 1101000 on the host.  So the actual
>>>>> uid against who the quota is counted is 1101000.  In another container,
>>>>> uid 1000 will be mapped to 1201000, and again quota will be counted against
>>>>> 1201000.
>>>>
>>>> Is it also an implications that we can examine do container quota or not
>>>> based on the uid/gid number?
>>>
>>> I'm sorry I don't understand the question.
>>
>> Sorry for my poor english.
>>
>>>
>>> As an attempt at an answer:  the quota code wouldn't change at all.  We would
>>> simply exploit the fact that uid 1000 in container1 has a real uid of 101100,
>>> which is different from the real uid 102100 assigned to uid 1000 in container2
>>> and from real uid 1000 (uid 1000 on the host).
>>
>> In that case, looks we only need to figure out how to let quota tools
>> works at container.
>> I'll build a new kernel with user_ns to give a try.
>   GETQUOTA or SETQUOTA quotactls should work just fine inside a container
> (for those quota-tools just need access to /proc/mounts). QUOTAON should
> also work for e.g. XFS or ext4 with hidden quota files. When quota files
> are visible in fs namespace (as for ext3 or so), things would be a bit
> tricky because they won't be possibly visible from container and QUOTAON
> needs that.

I still think if we can cache container dquot on memory to make this feature as simple as possible. :)

And also, quotacheck is the major issue I have faced previously, since we need a reasonable approach to calculate
and save the current inodes/blocks usage firstly.

> 
> Also with QUOTAON there is the principial problem that quotas either are or
> are not enabled for the whole filesystem.

IMHO, we could supply uid/gid quota for the whole filesystem only(i.e, the "/" rootfs), and we can support project
quota among sub-directories in the future if possible.

> So probably the only reasonable
> choice when you would like to supporot quotas in the container would be to
> have quotas enabled all the time, and inside the container, you would just
> set some quota limits or you won't...

I remember that ext4 has already supported quota as the first class, looks we can consider container quota same to that.

So we can ignore the quotacheck step, only focus on quota limits setup inside container?


Thanks for the teaching!
-Jeff

> 
> 								Honza

Re: container disk quota

[Jan Kara]

On Mon 04-06-12 21:35:06, Jeff Liu wrote:
> On 06/04/2012 05:42 PM, Jan Kara wrote:
> 
> > On Mon 04-06-12 12:46:49, Jeff Liu wrote:
> >> On 06/04/2012 10:57 AM, Serge Hallyn wrote:
> >>> Quoting Jeff Liu (jeff.liu@oracle.com):
> >>>> Hi Serge,
> >>>>
> >>>> On 06/02/2012 12:04 AM, Serge Hallyn wrote:
> >>>>
> >>>>> Quoting Jan Kara (jack@suse.cz):
> >>>>>>   Hello,
> >>>>>>
> >>>>>> On Wed 30-05-12 22:58:54, jeff.liu@oracle.com wrote:
> >>>>>>> According to glauber's comments regarding container disk quota, it should be binded to mount
> >>>>>>> namespace rather than cgroup.
> >>>>>>>
> >>>>>>> Per my try out, it works just fine by combining with userland quota utilitly in this way.
> >>>>>>> However, they are something has to be done at user tools too IMHO.
> >>>>>>>
> >>>>>>> Currently, the patchset is in very initial phase, I'd like to post it early to seek more
> >>>>>>> feedbacks from you guys.
> >>>>>>>
> >>>>>>> Hopefully I can clarify my ideas clearly.
> >>>>>>   So what I miss in this introductory email is some highlevel description
> >>>>>> like what is the desired functionality you try to implement and what is it
> >>>>>> good for. Looking at the examples below, it seems you want to be able to
> >>>>>> set quota limits for namespace-uid (and also namespace-gid???) pairs, am I
> >>>>>> right?
> >>>>>>
> >>>>>>   If yes, then I would like to understand one thing: When writing to a
> >>>>>> file, used space is accounted to the owner of the file. Now how do we
> >>>>>> determine owning namespace? Do you implicitely assume that only processes
> >>>>>> from one namespace will be able to access the file?
> >>>>>>
> >>>>>> 								Honza
> >>>>>
> >>>>> Not having looked closely at the original patchset, let me ask - is this
> >>>>> feature going to be a freebie with Eric's usernamespace patches?
> >>>>
> >>>> It we can reach a consensus to bind quota on mount namespace for
> >>>> container or other things maybe.
> >>>> I think it definitely should depends on user namespace.
> >>>>
> >>>>>
> >>>>> There, a container can be started in its own user namespace.  It's uid
> >>>>> 1000 will be mapped to something like 1101000 on the host.  So the actual
> >>>>> uid against who the quota is counted is 1101000.  In another container,
> >>>>> uid 1000 will be mapped to 1201000, and again quota will be counted against
> >>>>> 1201000.
> >>>>
> >>>> Is it also an implications that we can examine do container quota or not
> >>>> based on the uid/gid number?
> >>>
> >>> I'm sorry I don't understand the question.
> >>
> >> Sorry for my poor english.
> >>
> >>>
> >>> As an attempt at an answer:  the quota code wouldn't change at all.  We would
> >>> simply exploit the fact that uid 1000 in container1 has a real uid of 101100,
> >>> which is different from the real uid 102100 assigned to uid 1000 in container2
> >>> and from real uid 1000 (uid 1000 on the host).
> >>
> >> In that case, looks we only need to figure out how to let quota tools
> >> works at container.
> >> I'll build a new kernel with user_ns to give a try.
> >   GETQUOTA or SETQUOTA quotactls should work just fine inside a container
> > (for those quota-tools just need access to /proc/mounts). QUOTAON should
> > also work for e.g. XFS or ext4 with hidden quota files. When quota files
> > are visible in fs namespace (as for ext3 or so), things would be a bit
> > tricky because they won't be possibly visible from container and QUOTAON
> > needs that.
> 
> I still think if we can cache container dquot on memory to make this
> feature as simple as possible. :)
  Sorry, I don't understand. Quota structures are cached in memory. Also
what would be simpler if you also do some caching in a container?

> And also, quotacheck is the major issue I have faced previously, since we need a reasonable approach to calculate
> and save the current inodes/blocks usage firstly.
  Yes, quotacheck inside a container is a problem. But similarly as with
quotaon(8), I think such global operation should rather be done outside.
 
> > Also with QUOTAON there is the principial problem that quotas either are or
> > are not enabled for the whole filesystem.
> 
> IMHO, we could supply uid/gid quota for the whole filesystem only(i.e,
> the "/" rootfs), and we can support project quota among sub-directories
> in the future if possible.
> 
> > So probably the only reasonable
> > choice when you would like to supporot quotas in the container would be to
> > have quotas enabled all the time, and inside the container, you would just
> > set some quota limits or you won't...
> 
> I remember that ext4 has already supported quota as the first class,
> looks we can consider container quota same to that.
> 
> So we can ignore the quotacheck step, only focus on quota limits setup
> inside container?
  Yes, that would be my suggestion.

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

Re: container disk quota

[Jeff Liu]

On 06/04/2012 09:56 PM, Jan Kara wrote:

> On Mon 04-06-12 21:35:06, Jeff Liu wrote:
>> On 06/04/2012 05:42 PM, Jan Kara wrote:
>>
>>> On Mon 04-06-12 12:46:49, Jeff Liu wrote:
>>>> On 06/04/2012 10:57 AM, Serge Hallyn wrote:
>>>>> Quoting Jeff Liu (jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org):
>>>>>> Hi Serge,
>>>>>>
>>>>>> On 06/02/2012 12:04 AM, Serge Hallyn wrote:
>>>>>>
>>>>>>> Quoting Jan Kara (jack-AlSwsSmVLrQ@public.gmane.org):
>>>>>>>>   Hello,
>>>>>>>>
>>>>>>>> On Wed 30-05-12 22:58:54, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
>>>>>>>>> According to glauber's comments regarding container disk quota, it should be binded to mount
>>>>>>>>> namespace rather than cgroup.
>>>>>>>>>
>>>>>>>>> Per my try out, it works just fine by combining with userland quota utilitly in this way.
>>>>>>>>> However, they are something has to be done at user tools too IMHO.
>>>>>>>>>
>>>>>>>>> Currently, the patchset is in very initial phase, I'd like to post it early to seek more
>>>>>>>>> feedbacks from you guys.
>>>>>>>>>
>>>>>>>>> Hopefully I can clarify my ideas clearly.
>>>>>>>>   So what I miss in this introductory email is some highlevel description
>>>>>>>> like what is the desired functionality you try to implement and what is it
>>>>>>>> good for. Looking at the examples below, it seems you want to be able to
>>>>>>>> set quota limits for namespace-uid (and also namespace-gid???) pairs, am I
>>>>>>>> right?
>>>>>>>>
>>>>>>>>   If yes, then I would like to understand one thing: When writing to a
>>>>>>>> file, used space is accounted to the owner of the file. Now how do we
>>>>>>>> determine owning namespace? Do you implicitely assume that only processes
>>>>>>>> from one namespace will be able to access the file?
>>>>>>>>
>>>>>>>> 								Honza
>>>>>>>
>>>>>>> Not having looked closely at the original patchset, let me ask - is this
>>>>>>> feature going to be a freebie with Eric's usernamespace patches?
>>>>>>
>>>>>> It we can reach a consensus to bind quota on mount namespace for
>>>>>> container or other things maybe.
>>>>>> I think it definitely should depends on user namespace.
>>>>>>
>>>>>>>
>>>>>>> There, a container can be started in its own user namespace.  It's uid
>>>>>>> 1000 will be mapped to something like 1101000 on the host.  So the actual
>>>>>>> uid against who the quota is counted is 1101000.  In another container,
>>>>>>> uid 1000 will be mapped to 1201000, and again quota will be counted against
>>>>>>> 1201000.
>>>>>>
>>>>>> Is it also an implications that we can examine do container quota or not
>>>>>> based on the uid/gid number?
>>>>>
>>>>> I'm sorry I don't understand the question.
>>>>
>>>> Sorry for my poor english.
>>>>
>>>>>
>>>>> As an attempt at an answer:  the quota code wouldn't change at all.  We would
>>>>> simply exploit the fact that uid 1000 in container1 has a real uid of 101100,
>>>>> which is different from the real uid 102100 assigned to uid 1000 in container2
>>>>> and from real uid 1000 (uid 1000 on the host).
>>>>
>>>> In that case, looks we only need to figure out how to let quota tools
>>>> works at container.
>>>> I'll build a new kernel with user_ns to give a try.
>>>   GETQUOTA or SETQUOTA quotactls should work just fine inside a container
>>> (for those quota-tools just need access to /proc/mounts). QUOTAON should
>>> also work for e.g. XFS or ext4 with hidden quota files. When quota files
>>> are visible in fs namespace (as for ext3 or so), things would be a bit
>>> tricky because they won't be possibly visible from container and QUOTAON
>>> needs that.
>>
>> I still think if we can cache container dquot on memory to make this
>> feature as simple as possible. :)
>   Sorry, I don't understand. Quota structures are cached in memory.

I means teaching Q_SETQUOTA routine, don't write those info to quota
file if it was issued from container in quotacheck stage.  Instead,
allocate a dquot object at memory and keep it until quotaoff or
container destory procedures maybe.

> Also what would be simpler if you also do some caching in a container?

Sorry, does it means do caching in quota files?
currently, I have no good idea in this point. :(

> 
>> And also, quotacheck is the major issue I have faced previously, since we need a reasonable approach to calculate
>> and save the current inodes/blocks usage firstly.
>   Yes, quotacheck inside a container is a problem. But similarly as with
> quotaon(8), I think such global operation should rather be done outside.
>  
>>> Also with QUOTAON there is the principial problem that quotas either are or
>>> are not enabled for the whole filesystem.
>>
>> IMHO, we could supply uid/gid quota for the whole filesystem only(i.e,
>> the "/" rootfs), and we can support project quota among sub-directories
>> in the future if possible.
>>
>>> So probably the only reasonable
>>> choice when you would like to supporot quotas in the container would be to
>>> have quotas enabled all the time, and inside the container, you would just
>>> set some quota limits or you won't...
>>
>> I remember that ext4 has already supported quota as the first class,
>> looks we can consider container quota same to that.
>>
>> So we can ignore the quotacheck step, only focus on quota limits setup
>> inside container?
>   Yes, that would be my suggestion.

Yeah, that would be fine.

Thanks,
-Jeff

> 
> 								Honza

Re: container disk quota

[Jeff Liu]

On 06/04/2012 10:55 PM, Jeff Liu wrote:

> On 06/04/2012 09:56 PM, Jan Kara wrote:
> 
>> On Mon 04-06-12 21:35:06, Jeff Liu wrote:
>>> On 06/04/2012 05:42 PM, Jan Kara wrote:
>>>
>>>> On Mon 04-06-12 12:46:49, Jeff Liu wrote:
>>>>> On 06/04/2012 10:57 AM, Serge Hallyn wrote:
>>>>>> Quoting Jeff Liu (jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org):
>>>>>>> Hi Serge,
>>>>>>>
>>>>>>> On 06/02/2012 12:04 AM, Serge Hallyn wrote:
>>>>>>>
>>>>>>>> Quoting Jan Kara (jack-AlSwsSmVLrQ@public.gmane.org):
>>>>>>>>>   Hello,
>>>>>>>>>
>>>>>>>>> On Wed 30-05-12 22:58:54, jeff.liu-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org wrote:
>>>>>>>>>> According to glauber's comments regarding container disk quota, it should be binded to mount
>>>>>>>>>> namespace rather than cgroup.
>>>>>>>>>>
>>>>>>>>>> Per my try out, it works just fine by combining with userland quota utilitly in this way.
>>>>>>>>>> However, they are something has to be done at user tools too IMHO.
>>>>>>>>>>
>>>>>>>>>> Currently, the patchset is in very initial phase, I'd like to post it early to seek more
>>>>>>>>>> feedbacks from you guys.
>>>>>>>>>>
>>>>>>>>>> Hopefully I can clarify my ideas clearly.
>>>>>>>>>   So what I miss in this introductory email is some highlevel description
>>>>>>>>> like what is the desired functionality you try to implement and what is it
>>>>>>>>> good for. Looking at the examples below, it seems you want to be able to
>>>>>>>>> set quota limits for namespace-uid (and also namespace-gid???) pairs, am I
>>>>>>>>> right?
>>>>>>>>>
>>>>>>>>>   If yes, then I would like to understand one thing: When writing to a
>>>>>>>>> file, used space is accounted to the owner of the file. Now how do we
>>>>>>>>> determine owning namespace? Do you implicitely assume that only processes
>>>>>>>>> from one namespace will be able to access the file?
>>>>>>>>>
>>>>>>>>> 								Honza
>>>>>>>>
>>>>>>>> Not having looked closely at the original patchset, let me ask - is this
>>>>>>>> feature going to be a freebie with Eric's usernamespace patches?
>>>>>>>
>>>>>>> It we can reach a consensus to bind quota on mount namespace for
>>>>>>> container or other things maybe.
>>>>>>> I think it definitely should depends on user namespace.
>>>>>>>
>>>>>>>>
>>>>>>>> There, a container can be started in its own user namespace.  It's uid
>>>>>>>> 1000 will be mapped to something like 1101000 on the host.  So the actual
>>>>>>>> uid against who the quota is counted is 1101000.  In another container,
>>>>>>>> uid 1000 will be mapped to 1201000, and again quota will be counted against
>>>>>>>> 1201000.
>>>>>>>
>>>>>>> Is it also an implications that we can examine do container quota or not
>>>>>>> based on the uid/gid number?
>>>>>>
>>>>>> I'm sorry I don't understand the question.
>>>>>
>>>>> Sorry for my poor english.
>>>>>
>>>>>>
>>>>>> As an attempt at an answer:  the quota code wouldn't change at all.  We would
>>>>>> simply exploit the fact that uid 1000 in container1 has a real uid of 101100,
>>>>>> which is different from the real uid 102100 assigned to uid 1000 in container2
>>>>>> and from real uid 1000 (uid 1000 on the host).
>>>>>
>>>>> In that case, looks we only need to figure out how to let quota tools
>>>>> works at container.
>>>>> I'll build a new kernel with user_ns to give a try.
>>>>   GETQUOTA or SETQUOTA quotactls should work just fine inside a container
>>>> (for those quota-tools just need access to /proc/mounts). QUOTAON should
>>>> also work for e.g. XFS or ext4 with hidden quota files. When quota files
>>>> are visible in fs namespace (as for ext3 or so), things would be a bit
>>>> tricky because they won't be possibly visible from container and QUOTAON
>>>> needs that.
>>>
>>> I still think if we can cache container dquot on memory to make this
>>> feature as simple as possible. :)
>>   Sorry, I don't understand. Quota structures are cached in memory.
> 
> I means teaching Q_SETQUOTA routine, don't write those info to quota
> file if it was issued from container in quotacheck stage.  Instead,
> allocate a dquot object at memory and keep it until quotaoff or
> container destory procedures maybe.

Sorry, I must misled you.
We can not save quota usage info to memory cache without changing the
quota tools.

Originally, I introduced a new format option(which is "lxc") to
quotacheck, etc...
so if they were issued with -F "lxc" option, those tools will not
performed combined with quota file path as usual.

Since we would not like to change the quota tools, so it is absolutely
wrong.


Thanks,
-Jeff

> 
>> Also what would be simpler if you also do some caching in a container?
> 
> Sorry, does it means do caching in quota files?
> currently, I have no good idea in this point. :(
> 
>>
>>> And also, quotacheck is the major issue I have faced previously, since we need a reasonable approach to calculate
>>> and save the current inodes/blocks usage firstly.
>>   Yes, quotacheck inside a container is a problem. But similarly as with
>> quotaon(8), I think such global operation should rather be done outside.
>>  
>>>> Also with QUOTAON there is the principial problem that quotas either are or
>>>> are not enabled for the whole filesystem.
>>>
>>> IMHO, we could supply uid/gid quota for the whole filesystem only(i.e,
>>> the "/" rootfs), and we can support project quota among sub-directories
>>> in the future if possible.
>>>
>>>> So probably the only reasonable
>>>> choice when you would like to supporot quotas in the container would be to
>>>> have quotas enabled all the time, and inside the container, you would just
>>>> set some quota limits or you won't...
>>>
>>> I remember that ext4 has already supported quota as the first class,
>>> looks we can consider container quota same to that.
>>>
>>> So we can ignore the quotacheck step, only focus on quota limits setup
>>> inside container?
>>   Yes, that would be my suggestion.
> 
> Yeah, that would be fine.
> 
> Thanks,
> -Jeff
> 
>>
>> 								Honza
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 10/12] container quota: bill container inodes alloc/free on ext4.

[Dave Chinner]

On Thu, May 31, 2012 at 05:14:50PM +0400, Glauber Costa wrote:
> On 05/31/2012 04:58 PM, Jeff Liu wrote:
> >>>  This should be inside dquot_free_inode().
> >Yeah, Ted also mentioned this before, all name space dquot bill routine
> >could be placed to dquot_xxx() for VFS quota user.
> >
> >However, AFAICS, they still have to be exported to other file systems
> >which have their own quota management module, like XFS.
> >
> >Thanks,
> >-Jeff
> >
> Do you have a personal need to have it working with xfs, or is the
> use you are pursuing achievable with the other filesystems?
> 
> Unless you have a reason yourself to use xfs, I'd say don't bother
> with it too much at the moment. get it working for the standard
> interface first.

Which means you'll end up with something that has to be rewritten to
support XFS. Do it right the first time.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com