* quota file question
@ 2010-03-09 4:57 Dmitry Monakhov
2010-03-09 12:25 ` Jan Kara
0 siblings, 1 reply; 2+ messages in thread
From: Dmitry Monakhov @ 2010-03-09 4:57 UTC (permalink / raw)
To: Jan Kara, linux-fsdevel
[-- Attachment #1: Type: text/plain, Size: 536 bytes --]
I'm always wonder why do we have to allocate space for quota-file
as normal file. Because of this we have to grab i_mutex on each
quota write. And still getting in to risk of ENOSPC. The later is
really annoying issue for me(when i run fsstress tests).
Let's live the i_mutex's story for another day and consider the
simplest one. Why don't we preallocate a space for quota file
during creation. In fact we may imagine some reasonable quota-file
size expectation (65538 * 64) = 4Mb, At least this may be sufficient
for most use-cases.
[-- Attachment #2: 0001-quota-prealloc-space-for-new-quota-file.patch --]
[-- Type: text/plain, Size: 3155 bytes --]
>From ef72d4ad7d23db734a7ce74b350cdd47a594130b Mon Sep 17 00:00:00 2001
From: Dmitry Monakhov <dmonakhov@openvz.org>
Date: Sun, 7 Mar 2010 20:08:51 +0300
Subject: [PATCH] quota: prealloc space for new quota file.
Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
---
convertquota.c | 4 ++--
quotacheck.c | 2 +-
quotaio.c | 15 ++++++++++++++-
quotaio.h | 2 +-
4 files changed, 18 insertions(+), 5 deletions(-)
diff --git a/convertquota.c b/convertquota.c
index 1572cb9..2304be6 100644
--- a/convertquota.c
+++ b/convertquota.c
@@ -287,7 +287,7 @@ static int convert_format(int type, struct mntent *mnt)
type2name(type), mnt->mnt_dir);
return -1;
}
- if (!(qn = new_io(mnt, type, QF_VFSV0))) {
+ if (!(qn = new_io(mnt, type, QF_VFSV0, 8192*1024))) {
errstr(_("Cannot create file for %ss for new format on %s: %s\n"),
type2name(type), mnt->mnt_dir, strerror(errno));
end_io(qo);
@@ -320,7 +320,7 @@ static int convert_endian(int type, struct mntent *mnt)
close(ofd);
return -1;
}
- if (!(qn = new_io(mnt, type, QF_VFSV0))) {
+ if (!(qn = new_io(mnt, type, QF_VFSV0, 8192*1024))) {
errstr(_("Cannot create file for %ss for new format on %s: %s\n"),
type2name(type), mnt->mnt_dir, strerror(errno));
close(ofd);
diff --git a/quotacheck.c b/quotacheck.c
index b5f7e2e..2b49820 100644
--- a/quotacheck.c
+++ b/quotacheck.c
@@ -794,7 +794,7 @@ static int dump_to_file(struct mntent *mnt, int type)
struct quota_handle *h;
debug(FL_DEBUG, _("Dumping gathered data for %ss.\n"), type2name(type));
- if (!(h = new_io(mnt, type, cfmt))) {
+ if (!(h = new_io(mnt, type, cfmt, 8192*1024))) {
errstr(_("Cannot initialize IO on new quotafile: %s\n"),
strerror(errno));
return -1;
diff --git a/quotaio.c b/quotaio.c
index d74a37d..540e89b 100644
--- a/quotaio.c
+++ b/quotaio.c
@@ -166,7 +166,7 @@ out_handle:
/*
* Create new quotafile of specified format on given filesystem
*/
-struct quota_handle *new_io(struct mntent *mnt, int type, int fmt)
+struct quota_handle *new_io(struct mntent *mnt, int type, int fmt, int prealloc)
{
char *qfname;
int fd;
@@ -212,6 +212,19 @@ struct quota_handle *new_io(struct mntent *mnt, int type, int fmt)
free(h);
goto out_fd;
}
+ if (prealloc) {
+ char zbuf[4096];
+ int len, ret;
+ memset(zbuf, 0, sizeof(zbuf));
+ while (prealloc) {
+ len = (sizeof(zbuf) < prealloc) ? sizeof(zbuf) : prealloc;
+ ret = write(fd, zbuf, len);
+ if (ret < 0)
+ goto out_fd;
+ prealloc -= len;
+ }
+ lseek(fd, 0, SEEK_SET);
+ }
return h;
out_fd:
close(fd);
diff --git a/quotaio.h b/quotaio.h
index f6eef50..c66db43 100644
--- a/quotaio.h
+++ b/quotaio.h
@@ -169,7 +169,7 @@ static inline void mark_quotafile_info_dirty(struct quota_handle *h)
struct quota_handle *init_io(struct mntent *mnt, int type, int fmt, int flags);
/* Create new quotafile of specified format on given filesystem */
-struct quota_handle *new_io(struct mntent *mnt, int type, int fmt);
+struct quota_handle *new_io(struct mntent *mnt, int type, int fmt, int prealloc);
/* Close quotafile */
int end_io(struct quota_handle *h);
--
1.6.3.3
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: quota file question
2010-03-09 4:57 quota file question Dmitry Monakhov
@ 2010-03-09 12:25 ` Jan Kara
0 siblings, 0 replies; 2+ messages in thread
From: Jan Kara @ 2010-03-09 12:25 UTC (permalink / raw)
To: Dmitry Monakhov; +Cc: Jan Kara, linux-fsdevel
On Tue 09-03-10 07:57:59, Dmitry Monakhov wrote:
> I'm always wonder why do we have to allocate space for quota-file
> as normal file. Because of this we have to grab i_mutex on each
> quota write.
Yes, this isn't nice and it would be good to make quota files hidden
"system inodes" - like journal inode, etc. But then you need to implement
quotacheck(8) functionality in fsck and create new interface for
repquota(8).
> And still getting in to risk of ENOSPC. The later is
> really annoying issue for me(when i run fsstress tests).
Well, it might be annoying for fsstress tests but is it a problem for
real usage? Space for quota structure gets preallocated as soon as that
user creates his first file or has some quota limit set. So this usually
happens as soon as user is "created" in the system and at that time failing
because of ENOSPC is fine.
To make your fsstress test happy, you may force allocation of space for
quotafile by just setting some limit to 1000 users and then clearing it
again (quota file does not shrink).
> Let's live the i_mutex's story for another day and consider the
> simplest one. Why don't we preallocate a space for quota file
> during creation. In fact we may imagine some reasonable quota-file
> size expectation (65538 * 64) = 4Mb, At least this may be sufficient
> for most use-cases.
I'm sorry but I don't like this. If you really need to preallocate space
for quota file (which should be rare), you can do that as I've described
above and not by introducing some ad-hoc allocations to quota-tools...
Honza
> From ef72d4ad7d23db734a7ce74b350cdd47a594130b Mon Sep 17 00:00:00 2001
> From: Dmitry Monakhov <dmonakhov@openvz.org>
> Date: Sun, 7 Mar 2010 20:08:51 +0300
> Subject: [PATCH] quota: prealloc space for new quota file.
>
>
> Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
> ---
> convertquota.c | 4 ++--
> quotacheck.c | 2 +-
> quotaio.c | 15 ++++++++++++++-
> quotaio.h | 2 +-
> 4 files changed, 18 insertions(+), 5 deletions(-)
>
> diff --git a/convertquota.c b/convertquota.c
> index 1572cb9..2304be6 100644
> --- a/convertquota.c
> +++ b/convertquota.c
> @@ -287,7 +287,7 @@ static int convert_format(int type, struct mntent *mnt)
> type2name(type), mnt->mnt_dir);
> return -1;
> }
> - if (!(qn = new_io(mnt, type, QF_VFSV0))) {
> + if (!(qn = new_io(mnt, type, QF_VFSV0, 8192*1024))) {
> errstr(_("Cannot create file for %ss for new format on %s: %s\n"),
> type2name(type), mnt->mnt_dir, strerror(errno));
> end_io(qo);
> @@ -320,7 +320,7 @@ static int convert_endian(int type, struct mntent *mnt)
> close(ofd);
> return -1;
> }
> - if (!(qn = new_io(mnt, type, QF_VFSV0))) {
> + if (!(qn = new_io(mnt, type, QF_VFSV0, 8192*1024))) {
> errstr(_("Cannot create file for %ss for new format on %s: %s\n"),
> type2name(type), mnt->mnt_dir, strerror(errno));
> close(ofd);
> diff --git a/quotacheck.c b/quotacheck.c
> index b5f7e2e..2b49820 100644
> --- a/quotacheck.c
> +++ b/quotacheck.c
> @@ -794,7 +794,7 @@ static int dump_to_file(struct mntent *mnt, int type)
> struct quota_handle *h;
>
> debug(FL_DEBUG, _("Dumping gathered data for %ss.\n"), type2name(type));
> - if (!(h = new_io(mnt, type, cfmt))) {
> + if (!(h = new_io(mnt, type, cfmt, 8192*1024))) {
> errstr(_("Cannot initialize IO on new quotafile: %s\n"),
> strerror(errno));
> return -1;
> diff --git a/quotaio.c b/quotaio.c
> index d74a37d..540e89b 100644
> --- a/quotaio.c
> +++ b/quotaio.c
> @@ -166,7 +166,7 @@ out_handle:
> /*
> * Create new quotafile of specified format on given filesystem
> */
> -struct quota_handle *new_io(struct mntent *mnt, int type, int fmt)
> +struct quota_handle *new_io(struct mntent *mnt, int type, int fmt, int prealloc)
> {
> char *qfname;
> int fd;
> @@ -212,6 +212,19 @@ struct quota_handle *new_io(struct mntent *mnt, int type, int fmt)
> free(h);
> goto out_fd;
> }
> + if (prealloc) {
> + char zbuf[4096];
> + int len, ret;
> + memset(zbuf, 0, sizeof(zbuf));
> + while (prealloc) {
> + len = (sizeof(zbuf) < prealloc) ? sizeof(zbuf) : prealloc;
> + ret = write(fd, zbuf, len);
> + if (ret < 0)
> + goto out_fd;
> + prealloc -= len;
> + }
> + lseek(fd, 0, SEEK_SET);
> + }
> return h;
> out_fd:
> close(fd);
> diff --git a/quotaio.h b/quotaio.h
> index f6eef50..c66db43 100644
> --- a/quotaio.h
> +++ b/quotaio.h
> @@ -169,7 +169,7 @@ static inline void mark_quotafile_info_dirty(struct quota_handle *h)
> struct quota_handle *init_io(struct mntent *mnt, int type, int fmt, int flags);
>
> /* Create new quotafile of specified format on given filesystem */
> -struct quota_handle *new_io(struct mntent *mnt, int type, int fmt);
> +struct quota_handle *new_io(struct mntent *mnt, int type, int fmt, int prealloc);
>
> /* Close quotafile */
> int end_io(struct quota_handle *h);
> --
> 1.6.3.3
>
--
Jan Kara <jack@suse.cz>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2010-03-09 12:25 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-03-09 4:57 quota file question Dmitry Monakhov
2010-03-09 12:25 ` Jan Kara
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).