* [RFC] mkfs config file bikeshed now!
@ 2018-02-26 22:42 Darrick J. Wong
2018-02-26 23:42 ` Eric Sandeen
` (2 more replies)
0 siblings, 3 replies; 15+ messages in thread
From: Darrick J. Wong @ 2018-02-26 22:42 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: Eric Sandeen, Dave Chinner, Christoph Hellwig, linux-xfs
On Thu, Feb 22, 2018 at 05:59:09PM -0800, Darrick J. Wong wrote:
> > Clearly. Its still a nice sensible feature to address any defaults,
> > and now that I see chinner/mkfs-refactor is merged I'll go work
> > off of that and post patches again soon.
I'll help you start the config file format bikeshedding:
So here at $employer we have to support a bunch of different base system
and kernel configurations. Some of our customers run heterogeneous
environments where they have to be able to go back and forth between
recent and old systems (stock OL6.9 <-> OL7 + UEK4) whereas others only
care about formatting to whatever runs on the deployed system. Right
now we ship xfsprogs 4.5 with very conservative defaults, but this has
become a hassle to keep patching mkfs, to remember which option sets
correspond with which config, and to communicate that whole mess to
customers. Shipping multiple xfsprogs packages seems gross.
What we'd prefer to do is ship a single up to date xfsprogs package
along with a mkfs config file containing distro-supported filesystem
profiles. As a side note, if there's no config file then we'd use
whatever defaults are coded into xfs_mkfs.c.
For example, say we had a /etc/xfs.conf file shipping with xfsprogs:
[defaults]
metadata.crc = false
naming.ftype = false
[rhel7]
metadata.crc = true
[uek3]
naming.ftype = true
[uek4]
metadata.crc = true
naming.ftype = true
metdata.finobt = true
[experimental]
metadata.rmapbt = true
metadata.reflink = true
inode.sparse = true
data.agcount = 32
The "defaults" section enables the distro vendor to patch in whatever
default config they want to support. In the example above the distro is
not yet ready to support ftype or v5 filesystems everywhere because they
want maximum compatibility, even though xfsprogs (by this point in the
future) enables them by default.
# mkfs.xfs /dev/sda <-- formats a boring v4 filesystem
However, if a customer only runs "uek4" on their infrastructure, they
might want to take advantage of the superior performance of ftype and
finobt. They can elect to trade compatibility for speed:
# mkfs.xfs -t uek4 /dev/sda <-- v5 fs with ftype and finobt
The truly crazy can set their warranty cards on fire:
# mkfs.xfs -t experimental /dev/sda
Under this scheme, we take the suboption names directly from the command
line parser and expand the section names in some fashion:
b -> block
d -> data
i -> inode
l -> log
m -> metadata
n -> naming
r -> realtime
s -> sector
And throw in "label" and "protofile" for -L and -p.
Note that a libconfig file (Dave suggested libconfig ages ago) might
look more like this:
defaults {
metadata {
crc = false
}
naming {
ftype = false
}
}
rhel7 {
metadata {
crc = true
}
}
uek4 {
metadata {
crc = true
finobt = true
}
naming {
ftype = true
}
}
experimental {
metadata {
rmapbt = true
reflink = true
}
inode {
sparse = true
}
data {
agcount = 32
}
}
One downside of the libconfig API unfortunately is that we'd have to
manually iterate every option group/line to detect garbage / unknown key
values like:
broken {
metadata {
klsldgasldgjslgdajldsjgaldsgs = 98272352
}
}
since there's no way to give the parser a higher level schema that
constricts the allowable keywords. But that's (easily/clumsily) worked
around.
Ok, what does everyone else think?
--D
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now!
2018-02-26 22:42 [RFC] mkfs config file bikeshed now! Darrick J. Wong
@ 2018-02-26 23:42 ` Eric Sandeen
2018-02-26 23:56 ` Darrick J. Wong
2018-02-26 23:43 ` Eric Sandeen
2018-02-27 0:01 ` Luis R. Rodriguez
2 siblings, 1 reply; 15+ messages in thread
From: Eric Sandeen @ 2018-02-26 23:42 UTC (permalink / raw)
To: Darrick J. Wong, Luis R. Rodriguez
Cc: Dave Chinner, Christoph Hellwig, linux-xfs
On 2/26/18 4:42 PM, Darrick J. Wong wrote:
> On Thu, Feb 22, 2018 at 05:59:09PM -0800, Darrick J. Wong wrote:
>
>>> Clearly. Its still a nice sensible feature to address any defaults,
>>> and now that I see chinner/mkfs-refactor is merged I'll go work
>>> off of that and post patches again soon.
>
> I'll help you start the config file format bikeshedding:
>
> So here at $employer we have to support a bunch of different base system
> and kernel configurations. Some of our customers run heterogeneous
> environments where they have to be able to go back and forth between
> recent and old systems (stock OL6.9 <-> OL7 + UEK4) whereas others only
> care about formatting to whatever runs on the deployed system. Right
> now we ship xfsprogs 4.5 with very conservative defaults, but this has
> become a hassle to keep patching mkfs, to remember which option sets
> correspond with which config, and to communicate that whole mess to
> customers. Shipping multiple xfsprogs packages seems gross.
>
> What we'd prefer to do is ship a single up to date xfsprogs package
> along with a mkfs config file containing distro-supported filesystem
> profiles. As a side note, if there's no config file then we'd use
> whatever defaults are coded into xfs_mkfs.c.
I think that is an extremely important note, and the only sane behavior. :)
> For example, say we had a /etc/xfs.conf file shipping with xfsprogs:
/etc/mkfs.xfs.conf please, this configures mkfs and nothing else, right?
>
> [defaults]
> metadata.crc = false
> naming.ftype = false
>
> [rhel7]
> metadata.crc = true
>
> [uek3]
> naming.ftype = true
>
> [uek4]
> metadata.crc = true
> naming.ftype = true
> metdata.finobt = true
>
> [experimental]
> metadata.rmapbt = true
> metadata.reflink = true
> inode.sparse = true
> data.agcount = 32
>
> The "defaults" section enables the distro vendor to patch in whatever
> default config they want to support. In the example above the distro is
> not yet ready to support ftype or v5 filesystems everywhere because they
> want maximum compatibility, even though xfsprogs (by this point in the
> future) enables them by default.
Do the subsequent sections each independently specify the behavior or do they
build on the defaults? I'd prefer the former (do not build on [defaults]),
I think.
> # mkfs.xfs /dev/sda <-- formats a boring v4 filesystem
IIRC Dave had a patch that emitted what was being used for defaults if
not the code - also IIRC I removed it but we should put that back
in with any config file parsing, I think.
"Using defaults from config file [defaults]" or somesuch.
I had also asked whether upstream would ship a config file at all. I'd suggest
that we should not, because we wouldn't do anything other than restate
code defaults, and I'm not interested in keeping that consistent.
(OTOH a mkfs.xfs --config-file-defaults to spit out a code-defaults
configfile for editing might be neato) ;)
-Eric
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now!
2018-02-26 22:42 [RFC] mkfs config file bikeshed now! Darrick J. Wong
2018-02-26 23:42 ` Eric Sandeen
@ 2018-02-26 23:43 ` Eric Sandeen
2018-02-27 0:01 ` Luis R. Rodriguez
2 siblings, 0 replies; 15+ messages in thread
From: Eric Sandeen @ 2018-02-26 23:43 UTC (permalink / raw)
To: Darrick J. Wong, Luis R. Rodriguez
Cc: Dave Chinner, Christoph Hellwig, linux-xfs
On 2/26/18 4:42 PM, Darrick J. Wong wrote:
> For example, say we had a /etc/xfs.conf file shipping with xfsprogs:
>
> [defaults]
> metadata.crc = false
> naming.ftype = false
>
> [rhel7]
> metadata.crc = true
Oh, one more thing. Let's not arbitrarily decide that the commandline
uses 1/0 but the configfile uses true/false. I'd really like as much
1:1 consistency here as possible.
-Eric
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now!
2018-02-26 23:42 ` Eric Sandeen
@ 2018-02-26 23:56 ` Darrick J. Wong
2018-02-27 0:01 ` Eric Sandeen
2018-02-27 0:24 ` Eric Sandeen
0 siblings, 2 replies; 15+ messages in thread
From: Darrick J. Wong @ 2018-02-26 23:56 UTC (permalink / raw)
To: Eric Sandeen
Cc: Luis R. Rodriguez, Dave Chinner, Christoph Hellwig, linux-xfs
On Mon, Feb 26, 2018 at 05:42:15PM -0600, Eric Sandeen wrote:
>
>
> On 2/26/18 4:42 PM, Darrick J. Wong wrote:
> > On Thu, Feb 22, 2018 at 05:59:09PM -0800, Darrick J. Wong wrote:
> >
> >>> Clearly. Its still a nice sensible feature to address any defaults,
> >>> and now that I see chinner/mkfs-refactor is merged I'll go work
> >>> off of that and post patches again soon.
> >
> > I'll help you start the config file format bikeshedding:
> >
> > So here at $employer we have to support a bunch of different base system
> > and kernel configurations. Some of our customers run heterogeneous
> > environments where they have to be able to go back and forth between
> > recent and old systems (stock OL6.9 <-> OL7 + UEK4) whereas others only
> > care about formatting to whatever runs on the deployed system. Right
> > now we ship xfsprogs 4.5 with very conservative defaults, but this has
> > become a hassle to keep patching mkfs, to remember which option sets
> > correspond with which config, and to communicate that whole mess to
> > customers. Shipping multiple xfsprogs packages seems gross.
> >
> > What we'd prefer to do is ship a single up to date xfsprogs package
> > along with a mkfs config file containing distro-supported filesystem
> > profiles. As a side note, if there's no config file then we'd use
> > whatever defaults are coded into xfs_mkfs.c.
>
> I think that is an extremely important note, and the only sane behavior. :)
>
> > For example, say we had a /etc/xfs.conf file shipping with xfsprogs:
>
> /etc/mkfs.xfs.conf please, this configures mkfs and nothing else, right?
Ok. I mean, I might let that be $sysconfdir/mkfs.xfs.conf or something,
but I agree about the filename component of the path.
> >
> > [defaults]
> > metadata.crc = false
> > naming.ftype = false
> >
> > [rhel7]
> > metadata.crc = true
>
> Oh, one more thing. Let's not arbitrarily decide that the commandline
> uses 1/0 but the configfile uses true/false. I'd really like as much
> 1:1 consistency here as possible.
Oops. I of course meant 0 / 1 here, since the goal is to use the
existing mkfs parsing system here.
> > [uek3]
> > naming.ftype = true
> >
> > [uek4]
> > metadata.crc = true
> > naming.ftype = true
> > metdata.finobt = true
> >
> > [experimental]
> > metadata.rmapbt = true
> > metadata.reflink = true
> > inode.sparse = true
> > data.agcount = 32
> >
> > The "defaults" section enables the distro vendor to patch in whatever
> > default config they want to support. In the example above the distro is
> > not yet ready to support ftype or v5 filesystems everywhere because they
> > want maximum compatibility, even though xfsprogs (by this point in the
> > future) enables them by default.
>
> Do the subsequent sections each independently specify the behavior or do they
> build on the defaults? I'd prefer the former (do not build on [defaults]),
> I think.
Not sure either. It would be useful to let the non-defaults profile
inherit off of the defaults profile, but otoh to an administrator it's a
lot clearer what they're getting with a profile if we force the config
file writer to specify the entire configuration.
Hmmm, I am drifting towards "do not build on defaults".
> > # mkfs.xfs /dev/sda <-- formats a boring v4 filesystem
>
> IIRC Dave had a patch that emitted what was being used for defaults if
> not the code - also IIRC I removed it but we should put that back
> in with any config file parsing, I think.
>
> "Using defaults from config file [defaults]" or somesuch.
Yes. How about:
"Using profile 'rhel7' from /etc/mkfs.xfs.conf."
> I had also asked whether upstream would ship a config file at all. I'd suggest
> that we should not, because we wouldn't do anything other than restate
> code defaults, and I'm not interested in keeping that consistent.
I don't see much of a point since we'd be shipping
# mkfs.xfs --config-file-defaults > /etc/mkfs.xfs.conf
> (OTOH a mkfs.xfs --config-file-defaults to spit out a code-defaults
> configfile for editing might be neato) ;)
--generate-config-file?
--D
> -Eric
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now!
2018-02-26 22:42 [RFC] mkfs config file bikeshed now! Darrick J. Wong
2018-02-26 23:42 ` Eric Sandeen
2018-02-26 23:43 ` Eric Sandeen
@ 2018-02-27 0:01 ` Luis R. Rodriguez
2018-02-27 0:10 ` Eric Sandeen
2 siblings, 1 reply; 15+ messages in thread
From: Luis R. Rodriguez @ 2018-02-27 0:01 UTC (permalink / raw)
To: Darrick J. Wong, Tso Ted
Cc: Luis R. Rodriguez, Eric Sandeen, Dave Chinner, Christoph Hellwig,
Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs
On Mon, Feb 26, 2018 at 02:42:24PM -0800, Darrick J. Wong wrote:
> Note that a libconfig file (Dave suggested libconfig ages ago) might
> look more like this:
No, Dave *only hinted* towards libconfig ;)
I ran into grotesque issues with libconfig, and quickly moved to evaluate
libini_config (used for SSSD) as an alternative. IMHO it had the best
and up to date implementation, had great community traction, and I saw
support for it on a slew of old older distros.
I also evaluated Ted Tso's e2fprogs profile parser (based on MIT kerberos), and
it turns out that its 14.86% the size of libini_config.so.5.2.0, which is an
incredible bonus. It also supports to work with a slew of different OSes (which
we probably don't care for), but the gains in size are perhaps worth
considering. That and the fact that it will likely build on probably even
more ancient distros. The e2fsprogs profile parser also lacks uint64_t support
but adding support is very trivial. Another thing with e2fprogs's profile
parser is we'd have to come to an agreement with Ted and the e2fsprogs
community on sharing the library code. He seems to be open to that if we go
this route. To help I already did a lot of the work of extracting what we'd
need and also threw in a demo program:
https://gitlab.com/mcgrof/libekprofile
Similar review / demo is available with the other libraries:
https://gitlab.com/mcgrof/libconfig-int64-issues
https://gitlab.com/mcgrof/libini_config-demo
For boring details on further evaluation with other filesystems and what they
do and nitty gritty details of the above evaluation you can read my paper [0].
I even had evaluated NFS's configuration parser... at this point I was surprised
everyone keeps re-inventing the wheel...
One reason to consider sharing also is to help other filesystems as they
likely will run into similar requirements. Why does everyone keep re-inventing
this simple wheel on filesystem configuration parsers? Yes I suspect btrfs may
want something similar.
So in terms of bikeshedding -- its probably important also to at least zero down on
a library we want to work with. The rest I suspect will actually be pretty
trivial bikeshedding.
FWIW both libini_config and the e2fsprogs profile parser could cope with
namespaces on the configuration file as well. The syntax just varies
slightly. IIRC with e2fsprogs profile parsers we could end up with something
like:
[defaults]
foo=0
bar=0
[stuff]
some_release = {
foo=1
bar=2
}
new_release = {
foo=2
}
Meanwhile I think with libini_config it was something like:
[defaults]
foo=0
bar=0
[stuff/some_release]
foo=1
bar=2
[stuff/new_release]
foo=2
[0] https://gitlab.com/mcgrof/filesystem-configuration-paper/blob/master/paper.pdf
Luis
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now!
2018-02-26 23:56 ` Darrick J. Wong
@ 2018-02-27 0:01 ` Eric Sandeen
2018-02-27 0:24 ` Eric Sandeen
1 sibling, 0 replies; 15+ messages in thread
From: Eric Sandeen @ 2018-02-27 0:01 UTC (permalink / raw)
To: Darrick J. Wong
Cc: Luis R. Rodriguez, Dave Chinner, Christoph Hellwig, linux-xfs
On 2/26/18 5:56 PM, Darrick J. Wong wrote:
>> (OTOH a mkfs.xfs --config-file-defaults to spit out a code-defaults
>> configfile for editing might be neato) ;)
> --generate-config-file?
Yeah that's what I meant. I don't think it's that important, tho.
-Eric
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now!
2018-02-27 0:01 ` Luis R. Rodriguez
@ 2018-02-27 0:10 ` Eric Sandeen
2018-02-27 0:15 ` Luis R. Rodriguez
0 siblings, 1 reply; 15+ messages in thread
From: Eric Sandeen @ 2018-02-27 0:10 UTC (permalink / raw)
To: Luis R. Rodriguez, Darrick J. Wong, Tso Ted
Cc: Dave Chinner, Christoph Hellwig, Jan Kara, NeilBrown,
Jeff Mahoney, linux-xfs
On 2/26/18 6:01 PM, Luis R. Rodriguez wrote:
> IIRC with e2fsprogs profile parsers we could end up with something
> like:
>
> [defaults]
> foo=0
> bar=0
> [stuff]
> some_release = {
> foo=1
> bar=2
> }
> new_release = {
> foo=2
> }
I can't tell what that means. What's "stuff?" Why would we need this sort of
nesting?
I /really/ want to adhere to the KISS principle here as much as possible.
The last thing I want is for the config file structure to make it difficult
for admins to understand, and easy to mis-write, resulting in unexpected
configs deployed...
There are all kinds of clever things we could do (see previous question about
inheritance) but config files are going to be write-once-use-many for the most
part, and I really think that complexity would be a net negative here. I'd
much rather have explicit cut and paste in each section than confusing nesting
of option specifications.
-Eric
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now!
2018-02-27 0:10 ` Eric Sandeen
@ 2018-02-27 0:15 ` Luis R. Rodriguez
2018-02-27 0:25 ` Darrick J. Wong
0 siblings, 1 reply; 15+ messages in thread
From: Luis R. Rodriguez @ 2018-02-27 0:15 UTC (permalink / raw)
To: Eric Sandeen
Cc: Darrick J. Wong, Tso Ted, Dave Chinner, Christoph Hellwig,
Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs
On Mon, Feb 26, 2018 at 4:10 PM, Eric Sandeen <sandeen@sandeen.net> wrote:
>
>
> On 2/26/18 6:01 PM, Luis R. Rodriguez wrote:
>> IIRC with e2fsprogs profile parsers we could end up with something
>> like:
>>
>> [defaults]
>> foo=0
>> bar=0
>> [stuff]
>> some_release = {
>> foo=1
>> bar=2
>> }
>> new_release = {
>> foo=2
>> }
>
> I can't tell what that means. What's "stuff?" Why would we need this sort of
> nesting?
I'll leave the crafty examples to Darrick as he had some complex ideas
in mind. On my part I was happy to not have to deal with the namespace
stuff -- however I realized some may want it, so I had to mention both
libraries libini_config and e2fsprogs profile parser do support it,
just in different ways.
Luis
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now!
2018-02-26 23:56 ` Darrick J. Wong
2018-02-27 0:01 ` Eric Sandeen
@ 2018-02-27 0:24 ` Eric Sandeen
1 sibling, 0 replies; 15+ messages in thread
From: Eric Sandeen @ 2018-02-27 0:24 UTC (permalink / raw)
To: Darrick J. Wong
Cc: Luis R. Rodriguez, Dave Chinner, Christoph Hellwig, linux-xfs
On 2/26/18 5:56 PM, Darrick J. Wong wrote:
>>> For example, say we had a /etc/xfs.conf file shipping with xfsprogs:
>> /etc/mkfs.xfs.conf please, this configures mkfs and nothing else, right?
> Ok. I mean, I might let that be $sysconfdir/mkfs.xfs.conf or something,
> but I agree about the filename component of the path.
>
*nod*
Also a $sysconfdir/mkfs.xfs.conf.d/ scanning scheme might be a
useful feature.
Or if we ban inheritance between sections (please), as darrick stated
offline maybe just allowing configfile specification on the commandline
so that you could point to /etc/xfs/mkfs/$FOO_OS.conf
The point would be that over-arching OS config file package could
drop in all the relevant config files for its defaults rather than
munging it all together in one config file. I.e. if ceph wants one
thing and glusterfs wants another they could each supply their own
mkfs.xfs config file w/o hassle.
-Eric
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now!
2018-02-27 0:15 ` Luis R. Rodriguez
@ 2018-02-27 0:25 ` Darrick J. Wong
2018-02-27 3:17 ` Theodore Ts'o
0 siblings, 1 reply; 15+ messages in thread
From: Darrick J. Wong @ 2018-02-27 0:25 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: Eric Sandeen, Tso Ted, Dave Chinner, Christoph Hellwig, Jan Kara,
NeilBrown, Jeff Mahoney, linux-xfs
On Mon, Feb 26, 2018 at 04:15:06PM -0800, Luis R. Rodriguez wrote:
> On Mon, Feb 26, 2018 at 4:10 PM, Eric Sandeen <sandeen@sandeen.net> wrote:
> >
> >
> > On 2/26/18 6:01 PM, Luis R. Rodriguez wrote:
> >> IIRC with e2fsprogs profile parsers we could end up with something
> >> like:
> >>
> >> [defaults]
> >> foo=0
> >> bar=0
> >> [stuff]
> >> some_release = {
> >> foo=1
> >> bar=2
> >> }
> >> new_release = {
> >> foo=2
> >> }
> >
> > I can't tell what that means. What's "stuff?" Why would we need this sort of
> > nesting?
>
> I'll leave the crafty examples to Darrick as he had some complex ideas
> in mind. On my part I was happy to not have to deal with the namespace
> stuff -- however I realized some may want it, so I had to mention both
> libraries libini_config and e2fsprogs profile parser do support it,
> just in different ways.
Well I /did/ have an even stupider parser in mind if segmented config
files got shot down...
/etc/xfs/mkfs/uek5.conf:
data.agcount 32
metadata.reflink 0
# mkfs.xfs -t uek5 /dev/sda
open("/etc/xfs/mkfs/${t_optarg}.conf")...
Then we can parse it with a simple fgets/strtok loop to tokenize the
input lines and feed them to the geometry arg parser. But I'll go read
your paper all the way through first before I scribble more. :)
--D
> Luis
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now!
2018-02-27 0:25 ` Darrick J. Wong
@ 2018-02-27 3:17 ` Theodore Ts'o
2018-02-27 3:45 ` Theodore Ts'o
2018-02-27 18:11 ` Luis R. Rodriguez
0 siblings, 2 replies; 15+ messages in thread
From: Theodore Ts'o @ 2018-02-27 3:17 UTC (permalink / raw)
To: Darrick J. Wong
Cc: Luis R. Rodriguez, Eric Sandeen, Dave Chinner, Christoph Hellwig,
Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs
So number one, unlike everyone else, I didn't reinvent the wheel. I
grabbed the same wheel I wrote for MIT Kerberos years before. :-)
Number two, for examples for I wanted the nesting, from MIT Kerberos:
[realms]
ATHENA.MIT.EDU = {
kdc = kerberos.mit.edu
kdc = kerberos-1.mit.edu
kdc = kerberos-2.mit.edu:750
admin_server = kerberos.mit.edu
master_kdc = kerberos.mit.edu
default_domain = mit.edu
}
EXAMPLE.COM = {
kdc = kerberos.example.com
kdc = kerberos-1.example.com
admin_server = kerberos.example.com
}
And from the file system world, from /etc/mke2fs.conf:
[fs_types]
ext3 = {
features = has_journal
}
ext4 = {
features = has_journal,extent,huge_file,flex_bg,metadata_csum,64bit,dir_nlink,extra_isize
inode_size = 256
}
small = {
blocksize = 1024
inode_size = 128
inode_ratio = 4096
}
...
It's been handy inside Google since we have different file system
configs for different use cases, so we do things like:
mke2fs -T foo-use /dev/sdX
and
mke2fs -T bar-use /dev/sdX
... where foo-use and bar-use would be subsections under fs_types.
But hey, to be clear, I'm not the one trying to claim everyone should
use my library (even though it's the best looking and above-average,
ala Lake Wobegon. :-)
- Ted
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now!
2018-02-27 3:17 ` Theodore Ts'o
@ 2018-02-27 3:45 ` Theodore Ts'o
2018-02-27 13:43 ` Jan Tulak
2018-02-27 18:11 ` Luis R. Rodriguez
1 sibling, 1 reply; 15+ messages in thread
From: Theodore Ts'o @ 2018-02-27 3:45 UTC (permalink / raw)
To: Darrick J. Wong
Cc: Luis R. Rodriguez, Eric Sandeen, Dave Chinner, Christoph Hellwig,
Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs
One other thing I'll mention is that it was really important to me
that it was very easy to look up values from the config file. So for
me it was a lot more than just the parser. So I can fetch a read a
config parameter from the config file like this:
profile_get_boolean(ctx->profile, "options", "no_optimize_extents",
0, 0, &c);
That's because I use this for a lot more than just file system tuning
parametres, but also for changing how e2fsck / mke2fs behaves. So
trying to parse all of the config file at startup time and having to
stash that in some global context structure is just lot of extra work
that isn't necessary with the profile library compared with other
libraries which are focused exclusively on the parsing aspect of
things (which is actually pretty trivial).
Maybe this doesn't matter for XFS, but other file system utilities may
find this issue to be something worthy of consideration when trying to
pick a config file library.
- Ted
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now!
2018-02-27 3:45 ` Theodore Ts'o
@ 2018-02-27 13:43 ` Jan Tulak
2018-02-27 22:14 ` Dave Chinner
0 siblings, 1 reply; 15+ messages in thread
From: Jan Tulak @ 2018-02-27 13:43 UTC (permalink / raw)
To: Theodore Ts'o
Cc: Darrick J. Wong, Luis R. Rodriguez, Eric Sandeen, Dave Chinner,
Christoph Hellwig, Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs
On Tue, Feb 27, 2018 at 4:45 AM, Theodore Ts'o <tytso@mit.edu> wrote:
> One other thing I'll mention is that it was really important to me
> that it was very easy to look up values from the config file. So for
> me it was a lot more than just the parser. So I can fetch a read a
> config parameter from the config file like this:
>
> profile_get_boolean(ctx->profile, "options", "no_optimize_extents",
> 0, 0, &c);
>
> That's because I use this for a lot more than just file system tuning
> parametres, but also for changing how e2fsck / mke2fs behaves. So
> trying to parse all of the config file at startup time and having to
> stash that in some global context structure is just lot of extra work
> that isn't necessary with the profile library compared with other
> libraries which are focused exclusively on the parsing aspect of
> things (which is actually pretty trivial).
>
> Maybe this doesn't matter for XFS, but other file system utilities may
> find this issue to be something worthy of consideration when trying to
> pick a config file library.
Neat. But yeah, I don't think the nesting would be useful for xfs.
Especially as I prefer the /etc/mkfs.xfs.d/ idea. It might make sense
to use something like this, instead of foo.bar style prefixing,
though.
/etc/mkfs.xfs.d/default:
[metadata]
crc = 0
[naming]
ftype = 0
Cheers,
Jan
>
> - Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now!
2018-02-27 3:17 ` Theodore Ts'o
2018-02-27 3:45 ` Theodore Ts'o
@ 2018-02-27 18:11 ` Luis R. Rodriguez
1 sibling, 0 replies; 15+ messages in thread
From: Luis R. Rodriguez @ 2018-02-27 18:11 UTC (permalink / raw)
To: Theodore Ts'o
Cc: Darrick J. Wong, Luis R. Rodriguez, Eric Sandeen, Dave Chinner,
Christoph Hellwig, Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs
On Mon, Feb 26, 2018 at 10:17:57PM -0500, Theodore Ts'o wrote:
> So number one, unlike everyone else, I didn't reinvent the wheel. I
> grabbed the same wheel I wrote for MIT Kerberos years before. :-)
Yes thank you for not re-inventing the wheel.
> But hey, to be clear, I'm not the one trying to claim everyone should
> use my library (even though it's the best looking and above-average,
> ala Lake Wobegon. :-)
I started out with libconfig. Clearly that's crap. As I waited for the facelift
of mkfs/xfs_mkfs.c I found libini_config and was supportive of it as the best
in the community given community traction and good code. As I poked further and
extracted e2fsprogs profile library into its own repo and sized it up against
libini_config, I'll let it be on me to say:
I think its *very* sensible for us to consider using it *provided* we're up to
co-maintain a future small shared library in the hopes others will stop also
re-inventing the wheel further.
Unless I hear otherwise I'll give e2fsprogs profile library a shot for my
next iteration under the assumption we can share the library long term.
Luis
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now!
2018-02-27 13:43 ` Jan Tulak
@ 2018-02-27 22:14 ` Dave Chinner
0 siblings, 0 replies; 15+ messages in thread
From: Dave Chinner @ 2018-02-27 22:14 UTC (permalink / raw)
To: Jan Tulak
Cc: Theodore Ts'o, Darrick J. Wong, Luis R. Rodriguez,
Eric Sandeen, Christoph Hellwig, Jan Kara, NeilBrown,
Jeff Mahoney, linux-xfs
On Tue, Feb 27, 2018 at 02:43:44PM +0100, Jan Tulak wrote:
> On Tue, Feb 27, 2018 at 4:45 AM, Theodore Ts'o <tytso@mit.edu> wrote:
> > One other thing I'll mention is that it was really important to me
> > that it was very easy to look up values from the config file. So for
> > me it was a lot more than just the parser. So I can fetch a read a
> > config parameter from the config file like this:
> >
> > profile_get_boolean(ctx->profile, "options", "no_optimize_extents",
> > 0, 0, &c);
That's not the way we've set up mkfs to deal with config file
defaults.
i.e. the config file creates a "default values" structure that gets
used when no CLI parameter was specified. IOWs, all we need the
config file to do is to be parsed into a pre-defined structure and
the rest of mkfs will already DTRT.
i.e. We don't need (or want) mkfs to be directly pulling random
values from config files depending on the current option context.
> > That's because I use this for a lot more than just file system tuning
> > parametres, but also for changing how e2fsck / mke2fs behaves. So
> > trying to parse all of the config file at startup time and having to
> > stash that in some global context structure is just lot of extra work
> > that isn't necessary with the profile library compared with other
> > libraries which are focused exclusively on the parsing aspect of
> > things (which is actually pretty trivial).
> >
> > Maybe this doesn't matter for XFS, but other file system utilities may
> > find this issue to be something worthy of consideration when trying to
> > pick a config file library.
>
> Neat. But yeah, I don't think the nesting would be useful for xfs.
> Especially as I prefer the /etc/mkfs.xfs.d/ idea. It might make sense
> to use something like this, instead of foo.bar style prefixing,
> though.
>
> /etc/mkfs.xfs.d/default:
>
> [metadata]
> crc = 0
>
> [naming]
> ftype = 0
Yup, I like the simplicity of this approach. One config per file,
with simple config groups that mirror the CLI grouping and options.
Simple for admins, simple for developers, simple to implement and
test.
We really don't need any more complexity than this, so lets not make
it any more complex than it needs to be - that mistake has already
been made in recent times with mkfs.xfs modifications, and I don't
want to dig us out of that hole again...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2018-02-27 22:14 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-02-26 22:42 [RFC] mkfs config file bikeshed now! Darrick J. Wong
2018-02-26 23:42 ` Eric Sandeen
2018-02-26 23:56 ` Darrick J. Wong
2018-02-27 0:01 ` Eric Sandeen
2018-02-27 0:24 ` Eric Sandeen
2018-02-26 23:43 ` Eric Sandeen
2018-02-27 0:01 ` Luis R. Rodriguez
2018-02-27 0:10 ` Eric Sandeen
2018-02-27 0:15 ` Luis R. Rodriguez
2018-02-27 0:25 ` Darrick J. Wong
2018-02-27 3:17 ` Theodore Ts'o
2018-02-27 3:45 ` Theodore Ts'o
2018-02-27 13:43 ` Jan Tulak
2018-02-27 22:14 ` Dave Chinner
2018-02-27 18:11 ` Luis R. Rodriguez
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).