* [RFC] mkfs config file bikeshed now!
@ 2018-02-26 22:42 Darrick J. Wong
2018-02-26 23:42 ` Eric Sandeen
` (2 more replies)
0 siblings, 3 replies; 15+ messages in thread
From: Darrick J. Wong @ 2018-02-26 22:42 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: Eric Sandeen, Dave Chinner, Christoph Hellwig, linux-xfs
On Thu, Feb 22, 2018 at 05:59:09PM -0800, Darrick J. Wong wrote:
> > Clearly. Its still a nice sensible feature to address any defaults,
> > and now that I see chinner/mkfs-refactor is merged I'll go work
> > off of that and post patches again soon.
I'll help you start the config file format bikeshedding:
So here at $employer we have to support a bunch of different base system
and kernel configurations. Some of our customers run heterogeneous
environments where they have to be able to go back and forth between
recent and old systems (stock OL6.9 <-> OL7 + UEK4) whereas others only
care about formatting to whatever runs on the deployed system. Right
now we ship xfsprogs 4.5 with very conservative defaults, but this has
become a hassle to keep patching mkfs, to remember which option sets
correspond with which config, and to communicate that whole mess to
customers. Shipping multiple xfsprogs packages seems gross.
What we'd prefer to do is ship a single up to date xfsprogs package
along with a mkfs config file containing distro-supported filesystem
profiles. As a side note, if there's no config file then we'd use
whatever defaults are coded into xfs_mkfs.c.
For example, say we had a /etc/xfs.conf file shipping with xfsprogs:
[defaults]
metadata.crc = false
naming.ftype = false
[rhel7]
metadata.crc = true
[uek3]
naming.ftype = true
[uek4]
metadata.crc = true
naming.ftype = true
metdata.finobt = true
[experimental]
metadata.rmapbt = true
metadata.reflink = true
inode.sparse = true
data.agcount = 32
The "defaults" section enables the distro vendor to patch in whatever
default config they want to support. In the example above the distro is
not yet ready to support ftype or v5 filesystems everywhere because they
want maximum compatibility, even though xfsprogs (by this point in the
future) enables them by default.
# mkfs.xfs /dev/sda <-- formats a boring v4 filesystem
However, if a customer only runs "uek4" on their infrastructure, they
might want to take advantage of the superior performance of ftype and
finobt. They can elect to trade compatibility for speed:
# mkfs.xfs -t uek4 /dev/sda <-- v5 fs with ftype and finobt
The truly crazy can set their warranty cards on fire:
# mkfs.xfs -t experimental /dev/sda
Under this scheme, we take the suboption names directly from the command
line parser and expand the section names in some fashion:
b -> block
d -> data
i -> inode
l -> log
m -> metadata
n -> naming
r -> realtime
s -> sector
And throw in "label" and "protofile" for -L and -p.
Note that a libconfig file (Dave suggested libconfig ages ago) might
look more like this:
defaults {
metadata {
crc = false
}
naming {
ftype = false
}
}
rhel7 {
metadata {
crc = true
}
}
uek4 {
metadata {
crc = true
finobt = true
}
naming {
ftype = true
}
}
experimental {
metadata {
rmapbt = true
reflink = true
}
inode {
sparse = true
}
data {
agcount = 32
}
}
One downside of the libconfig API unfortunately is that we'd have to
manually iterate every option group/line to detect garbage / unknown key
values like:
broken {
metadata {
klsldgasldgjslgdajldsjgaldsgs = 98272352
}
}
since there's no way to give the parser a higher level schema that
constricts the allowable keywords. But that's (easily/clumsily) worked
around.
Ok, what does everyone else think?
--D
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [RFC] mkfs config file bikeshed now! 2018-02-26 22:42 [RFC] mkfs config file bikeshed now! Darrick J. Wong @ 2018-02-26 23:42 ` Eric Sandeen 2018-02-26 23:56 ` Darrick J. Wong 2018-02-26 23:43 ` Eric Sandeen 2018-02-27 0:01 ` Luis R. Rodriguez 2 siblings, 1 reply; 15+ messages in thread From: Eric Sandeen @ 2018-02-26 23:42 UTC (permalink / raw) To: Darrick J. Wong, Luis R. Rodriguez Cc: Dave Chinner, Christoph Hellwig, linux-xfs On 2/26/18 4:42 PM, Darrick J. Wong wrote: > On Thu, Feb 22, 2018 at 05:59:09PM -0800, Darrick J. Wong wrote: > >>> Clearly. Its still a nice sensible feature to address any defaults, >>> and now that I see chinner/mkfs-refactor is merged I'll go work >>> off of that and post patches again soon. > > I'll help you start the config file format bikeshedding: > > So here at $employer we have to support a bunch of different base system > and kernel configurations. Some of our customers run heterogeneous > environments where they have to be able to go back and forth between > recent and old systems (stock OL6.9 <-> OL7 + UEK4) whereas others only > care about formatting to whatever runs on the deployed system. Right > now we ship xfsprogs 4.5 with very conservative defaults, but this has > become a hassle to keep patching mkfs, to remember which option sets > correspond with which config, and to communicate that whole mess to > customers. Shipping multiple xfsprogs packages seems gross. > > What we'd prefer to do is ship a single up to date xfsprogs package > along with a mkfs config file containing distro-supported filesystem > profiles. As a side note, if there's no config file then we'd use > whatever defaults are coded into xfs_mkfs.c. I think that is an extremely important note, and the only sane behavior. :) > For example, say we had a /etc/xfs.conf file shipping with xfsprogs: /etc/mkfs.xfs.conf please, this configures mkfs and nothing else, right? > > [defaults] > metadata.crc = false > naming.ftype = false > > [rhel7] > metadata.crc = true > > [uek3] > naming.ftype = true > > [uek4] > metadata.crc = true > naming.ftype = true > metdata.finobt = true > > [experimental] > metadata.rmapbt = true > metadata.reflink = true > inode.sparse = true > data.agcount = 32 > > The "defaults" section enables the distro vendor to patch in whatever > default config they want to support. In the example above the distro is > not yet ready to support ftype or v5 filesystems everywhere because they > want maximum compatibility, even though xfsprogs (by this point in the > future) enables them by default. Do the subsequent sections each independently specify the behavior or do they build on the defaults? I'd prefer the former (do not build on [defaults]), I think. > # mkfs.xfs /dev/sda <-- formats a boring v4 filesystem IIRC Dave had a patch that emitted what was being used for defaults if not the code - also IIRC I removed it but we should put that back in with any config file parsing, I think. "Using defaults from config file [defaults]" or somesuch. I had also asked whether upstream would ship a config file at all. I'd suggest that we should not, because we wouldn't do anything other than restate code defaults, and I'm not interested in keeping that consistent. (OTOH a mkfs.xfs --config-file-defaults to spit out a code-defaults configfile for editing might be neato) ;) -Eric ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now! 2018-02-26 23:42 ` Eric Sandeen @ 2018-02-26 23:56 ` Darrick J. Wong 2018-02-27 0:01 ` Eric Sandeen 2018-02-27 0:24 ` Eric Sandeen 0 siblings, 2 replies; 15+ messages in thread From: Darrick J. Wong @ 2018-02-26 23:56 UTC (permalink / raw) To: Eric Sandeen Cc: Luis R. Rodriguez, Dave Chinner, Christoph Hellwig, linux-xfs On Mon, Feb 26, 2018 at 05:42:15PM -0600, Eric Sandeen wrote: > > > On 2/26/18 4:42 PM, Darrick J. Wong wrote: > > On Thu, Feb 22, 2018 at 05:59:09PM -0800, Darrick J. Wong wrote: > > > >>> Clearly. Its still a nice sensible feature to address any defaults, > >>> and now that I see chinner/mkfs-refactor is merged I'll go work > >>> off of that and post patches again soon. > > > > I'll help you start the config file format bikeshedding: > > > > So here at $employer we have to support a bunch of different base system > > and kernel configurations. Some of our customers run heterogeneous > > environments where they have to be able to go back and forth between > > recent and old systems (stock OL6.9 <-> OL7 + UEK4) whereas others only > > care about formatting to whatever runs on the deployed system. Right > > now we ship xfsprogs 4.5 with very conservative defaults, but this has > > become a hassle to keep patching mkfs, to remember which option sets > > correspond with which config, and to communicate that whole mess to > > customers. Shipping multiple xfsprogs packages seems gross. > > > > What we'd prefer to do is ship a single up to date xfsprogs package > > along with a mkfs config file containing distro-supported filesystem > > profiles. As a side note, if there's no config file then we'd use > > whatever defaults are coded into xfs_mkfs.c. > > I think that is an extremely important note, and the only sane behavior. :) > > > For example, say we had a /etc/xfs.conf file shipping with xfsprogs: > > /etc/mkfs.xfs.conf please, this configures mkfs and nothing else, right? Ok. I mean, I might let that be $sysconfdir/mkfs.xfs.conf or something, but I agree about the filename component of the path. > > > > [defaults] > > metadata.crc = false > > naming.ftype = false > > > > [rhel7] > > metadata.crc = true > > Oh, one more thing. Let's not arbitrarily decide that the commandline > uses 1/0 but the configfile uses true/false. I'd really like as much > 1:1 consistency here as possible. Oops. I of course meant 0 / 1 here, since the goal is to use the existing mkfs parsing system here. > > [uek3] > > naming.ftype = true > > > > [uek4] > > metadata.crc = true > > naming.ftype = true > > metdata.finobt = true > > > > [experimental] > > metadata.rmapbt = true > > metadata.reflink = true > > inode.sparse = true > > data.agcount = 32 > > > > The "defaults" section enables the distro vendor to patch in whatever > > default config they want to support. In the example above the distro is > > not yet ready to support ftype or v5 filesystems everywhere because they > > want maximum compatibility, even though xfsprogs (by this point in the > > future) enables them by default. > > Do the subsequent sections each independently specify the behavior or do they > build on the defaults? I'd prefer the former (do not build on [defaults]), > I think. Not sure either. It would be useful to let the non-defaults profile inherit off of the defaults profile, but otoh to an administrator it's a lot clearer what they're getting with a profile if we force the config file writer to specify the entire configuration. Hmmm, I am drifting towards "do not build on defaults". > > # mkfs.xfs /dev/sda <-- formats a boring v4 filesystem > > IIRC Dave had a patch that emitted what was being used for defaults if > not the code - also IIRC I removed it but we should put that back > in with any config file parsing, I think. > > "Using defaults from config file [defaults]" or somesuch. Yes. How about: "Using profile 'rhel7' from /etc/mkfs.xfs.conf." > I had also asked whether upstream would ship a config file at all. I'd suggest > that we should not, because we wouldn't do anything other than restate > code defaults, and I'm not interested in keeping that consistent. I don't see much of a point since we'd be shipping # mkfs.xfs --config-file-defaults > /etc/mkfs.xfs.conf > (OTOH a mkfs.xfs --config-file-defaults to spit out a code-defaults > configfile for editing might be neato) ;) --generate-config-file? --D > -Eric > -- > To unsubscribe from this list: send the line "unsubscribe linux-xfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now! 2018-02-26 23:56 ` Darrick J. Wong @ 2018-02-27 0:01 ` Eric Sandeen 2018-02-27 0:24 ` Eric Sandeen 1 sibling, 0 replies; 15+ messages in thread From: Eric Sandeen @ 2018-02-27 0:01 UTC (permalink / raw) To: Darrick J. Wong Cc: Luis R. Rodriguez, Dave Chinner, Christoph Hellwig, linux-xfs On 2/26/18 5:56 PM, Darrick J. Wong wrote: >> (OTOH a mkfs.xfs --config-file-defaults to spit out a code-defaults >> configfile for editing might be neato) ;) > --generate-config-file? Yeah that's what I meant. I don't think it's that important, tho. -Eric ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now! 2018-02-26 23:56 ` Darrick J. Wong 2018-02-27 0:01 ` Eric Sandeen @ 2018-02-27 0:24 ` Eric Sandeen 1 sibling, 0 replies; 15+ messages in thread From: Eric Sandeen @ 2018-02-27 0:24 UTC (permalink / raw) To: Darrick J. Wong Cc: Luis R. Rodriguez, Dave Chinner, Christoph Hellwig, linux-xfs On 2/26/18 5:56 PM, Darrick J. Wong wrote: >>> For example, say we had a /etc/xfs.conf file shipping with xfsprogs: >> /etc/mkfs.xfs.conf please, this configures mkfs and nothing else, right? > Ok. I mean, I might let that be $sysconfdir/mkfs.xfs.conf or something, > but I agree about the filename component of the path. > *nod* Also a $sysconfdir/mkfs.xfs.conf.d/ scanning scheme might be a useful feature. Or if we ban inheritance between sections (please), as darrick stated offline maybe just allowing configfile specification on the commandline so that you could point to /etc/xfs/mkfs/$FOO_OS.conf The point would be that over-arching OS config file package could drop in all the relevant config files for its defaults rather than munging it all together in one config file. I.e. if ceph wants one thing and glusterfs wants another they could each supply their own mkfs.xfs config file w/o hassle. -Eric ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now! 2018-02-26 22:42 [RFC] mkfs config file bikeshed now! Darrick J. Wong 2018-02-26 23:42 ` Eric Sandeen @ 2018-02-26 23:43 ` Eric Sandeen 2018-02-27 0:01 ` Luis R. Rodriguez 2 siblings, 0 replies; 15+ messages in thread From: Eric Sandeen @ 2018-02-26 23:43 UTC (permalink / raw) To: Darrick J. Wong, Luis R. Rodriguez Cc: Dave Chinner, Christoph Hellwig, linux-xfs On 2/26/18 4:42 PM, Darrick J. Wong wrote: > For example, say we had a /etc/xfs.conf file shipping with xfsprogs: > > [defaults] > metadata.crc = false > naming.ftype = false > > [rhel7] > metadata.crc = true Oh, one more thing. Let's not arbitrarily decide that the commandline uses 1/0 but the configfile uses true/false. I'd really like as much 1:1 consistency here as possible. -Eric ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now! 2018-02-26 22:42 [RFC] mkfs config file bikeshed now! Darrick J. Wong 2018-02-26 23:42 ` Eric Sandeen 2018-02-26 23:43 ` Eric Sandeen @ 2018-02-27 0:01 ` Luis R. Rodriguez 2018-02-27 0:10 ` Eric Sandeen 2 siblings, 1 reply; 15+ messages in thread From: Luis R. Rodriguez @ 2018-02-27 0:01 UTC (permalink / raw) To: Darrick J. Wong, Tso Ted Cc: Luis R. Rodriguez, Eric Sandeen, Dave Chinner, Christoph Hellwig, Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs On Mon, Feb 26, 2018 at 02:42:24PM -0800, Darrick J. Wong wrote: > Note that a libconfig file (Dave suggested libconfig ages ago) might > look more like this: No, Dave *only hinted* towards libconfig ;) I ran into grotesque issues with libconfig, and quickly moved to evaluate libini_config (used for SSSD) as an alternative. IMHO it had the best and up to date implementation, had great community traction, and I saw support for it on a slew of old older distros. I also evaluated Ted Tso's e2fprogs profile parser (based on MIT kerberos), and it turns out that its 14.86% the size of libini_config.so.5.2.0, which is an incredible bonus. It also supports to work with a slew of different OSes (which we probably don't care for), but the gains in size are perhaps worth considering. That and the fact that it will likely build on probably even more ancient distros. The e2fsprogs profile parser also lacks uint64_t support but adding support is very trivial. Another thing with e2fprogs's profile parser is we'd have to come to an agreement with Ted and the e2fsprogs community on sharing the library code. He seems to be open to that if we go this route. To help I already did a lot of the work of extracting what we'd need and also threw in a demo program: https://gitlab.com/mcgrof/libekprofile Similar review / demo is available with the other libraries: https://gitlab.com/mcgrof/libconfig-int64-issues https://gitlab.com/mcgrof/libini_config-demo For boring details on further evaluation with other filesystems and what they do and nitty gritty details of the above evaluation you can read my paper [0]. I even had evaluated NFS's configuration parser... at this point I was surprised everyone keeps re-inventing the wheel... One reason to consider sharing also is to help other filesystems as they likely will run into similar requirements. Why does everyone keep re-inventing this simple wheel on filesystem configuration parsers? Yes I suspect btrfs may want something similar. So in terms of bikeshedding -- its probably important also to at least zero down on a library we want to work with. The rest I suspect will actually be pretty trivial bikeshedding. FWIW both libini_config and the e2fsprogs profile parser could cope with namespaces on the configuration file as well. The syntax just varies slightly. IIRC with e2fsprogs profile parsers we could end up with something like: [defaults] foo=0 bar=0 [stuff] some_release = { foo=1 bar=2 } new_release = { foo=2 } Meanwhile I think with libini_config it was something like: [defaults] foo=0 bar=0 [stuff/some_release] foo=1 bar=2 [stuff/new_release] foo=2 [0] https://gitlab.com/mcgrof/filesystem-configuration-paper/blob/master/paper.pdf Luis ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now! 2018-02-27 0:01 ` Luis R. Rodriguez @ 2018-02-27 0:10 ` Eric Sandeen 2018-02-27 0:15 ` Luis R. Rodriguez 0 siblings, 1 reply; 15+ messages in thread From: Eric Sandeen @ 2018-02-27 0:10 UTC (permalink / raw) To: Luis R. Rodriguez, Darrick J. Wong, Tso Ted Cc: Dave Chinner, Christoph Hellwig, Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs On 2/26/18 6:01 PM, Luis R. Rodriguez wrote: > IIRC with e2fsprogs profile parsers we could end up with something > like: > > [defaults] > foo=0 > bar=0 > [stuff] > some_release = { > foo=1 > bar=2 > } > new_release = { > foo=2 > } I can't tell what that means. What's "stuff?" Why would we need this sort of nesting? I /really/ want to adhere to the KISS principle here as much as possible. The last thing I want is for the config file structure to make it difficult for admins to understand, and easy to mis-write, resulting in unexpected configs deployed... There are all kinds of clever things we could do (see previous question about inheritance) but config files are going to be write-once-use-many for the most part, and I really think that complexity would be a net negative here. I'd much rather have explicit cut and paste in each section than confusing nesting of option specifications. -Eric ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now! 2018-02-27 0:10 ` Eric Sandeen @ 2018-02-27 0:15 ` Luis R. Rodriguez 2018-02-27 0:25 ` Darrick J. Wong 0 siblings, 1 reply; 15+ messages in thread From: Luis R. Rodriguez @ 2018-02-27 0:15 UTC (permalink / raw) To: Eric Sandeen Cc: Darrick J. Wong, Tso Ted, Dave Chinner, Christoph Hellwig, Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs On Mon, Feb 26, 2018 at 4:10 PM, Eric Sandeen <sandeen@sandeen.net> wrote: > > > On 2/26/18 6:01 PM, Luis R. Rodriguez wrote: >> IIRC with e2fsprogs profile parsers we could end up with something >> like: >> >> [defaults] >> foo=0 >> bar=0 >> [stuff] >> some_release = { >> foo=1 >> bar=2 >> } >> new_release = { >> foo=2 >> } > > I can't tell what that means. What's "stuff?" Why would we need this sort of > nesting? I'll leave the crafty examples to Darrick as he had some complex ideas in mind. On my part I was happy to not have to deal with the namespace stuff -- however I realized some may want it, so I had to mention both libraries libini_config and e2fsprogs profile parser do support it, just in different ways. Luis ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now! 2018-02-27 0:15 ` Luis R. Rodriguez @ 2018-02-27 0:25 ` Darrick J. Wong 2018-02-27 3:17 ` Theodore Ts'o 0 siblings, 1 reply; 15+ messages in thread From: Darrick J. Wong @ 2018-02-27 0:25 UTC (permalink / raw) To: Luis R. Rodriguez Cc: Eric Sandeen, Tso Ted, Dave Chinner, Christoph Hellwig, Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs On Mon, Feb 26, 2018 at 04:15:06PM -0800, Luis R. Rodriguez wrote: > On Mon, Feb 26, 2018 at 4:10 PM, Eric Sandeen <sandeen@sandeen.net> wrote: > > > > > > On 2/26/18 6:01 PM, Luis R. Rodriguez wrote: > >> IIRC with e2fsprogs profile parsers we could end up with something > >> like: > >> > >> [defaults] > >> foo=0 > >> bar=0 > >> [stuff] > >> some_release = { > >> foo=1 > >> bar=2 > >> } > >> new_release = { > >> foo=2 > >> } > > > > I can't tell what that means. What's "stuff?" Why would we need this sort of > > nesting? > > I'll leave the crafty examples to Darrick as he had some complex ideas > in mind. On my part I was happy to not have to deal with the namespace > stuff -- however I realized some may want it, so I had to mention both > libraries libini_config and e2fsprogs profile parser do support it, > just in different ways. Well I /did/ have an even stupider parser in mind if segmented config files got shot down... /etc/xfs/mkfs/uek5.conf: data.agcount 32 metadata.reflink 0 # mkfs.xfs -t uek5 /dev/sda open("/etc/xfs/mkfs/${t_optarg}.conf")... Then we can parse it with a simple fgets/strtok loop to tokenize the input lines and feed them to the geometry arg parser. But I'll go read your paper all the way through first before I scribble more. :) --D > Luis > -- > To unsubscribe from this list: send the line "unsubscribe linux-xfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now! 2018-02-27 0:25 ` Darrick J. Wong @ 2018-02-27 3:17 ` Theodore Ts'o 2018-02-27 3:45 ` Theodore Ts'o 2018-02-27 18:11 ` Luis R. Rodriguez 0 siblings, 2 replies; 15+ messages in thread From: Theodore Ts'o @ 2018-02-27 3:17 UTC (permalink / raw) To: Darrick J. Wong Cc: Luis R. Rodriguez, Eric Sandeen, Dave Chinner, Christoph Hellwig, Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs So number one, unlike everyone else, I didn't reinvent the wheel. I grabbed the same wheel I wrote for MIT Kerberos years before. :-) Number two, for examples for I wanted the nesting, from MIT Kerberos: [realms] ATHENA.MIT.EDU = { kdc = kerberos.mit.edu kdc = kerberos-1.mit.edu kdc = kerberos-2.mit.edu:750 admin_server = kerberos.mit.edu master_kdc = kerberos.mit.edu default_domain = mit.edu } EXAMPLE.COM = { kdc = kerberos.example.com kdc = kerberos-1.example.com admin_server = kerberos.example.com } And from the file system world, from /etc/mke2fs.conf: [fs_types] ext3 = { features = has_journal } ext4 = { features = has_journal,extent,huge_file,flex_bg,metadata_csum,64bit,dir_nlink,extra_isize inode_size = 256 } small = { blocksize = 1024 inode_size = 128 inode_ratio = 4096 } ... It's been handy inside Google since we have different file system configs for different use cases, so we do things like: mke2fs -T foo-use /dev/sdX and mke2fs -T bar-use /dev/sdX ... where foo-use and bar-use would be subsections under fs_types. But hey, to be clear, I'm not the one trying to claim everyone should use my library (even though it's the best looking and above-average, ala Lake Wobegon. :-) - Ted ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now! 2018-02-27 3:17 ` Theodore Ts'o @ 2018-02-27 3:45 ` Theodore Ts'o 2018-02-27 13:43 ` Jan Tulak 2018-02-27 18:11 ` Luis R. Rodriguez 1 sibling, 1 reply; 15+ messages in thread From: Theodore Ts'o @ 2018-02-27 3:45 UTC (permalink / raw) To: Darrick J. Wong Cc: Luis R. Rodriguez, Eric Sandeen, Dave Chinner, Christoph Hellwig, Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs One other thing I'll mention is that it was really important to me that it was very easy to look up values from the config file. So for me it was a lot more than just the parser. So I can fetch a read a config parameter from the config file like this: profile_get_boolean(ctx->profile, "options", "no_optimize_extents", 0, 0, &c); That's because I use this for a lot more than just file system tuning parametres, but also for changing how e2fsck / mke2fs behaves. So trying to parse all of the config file at startup time and having to stash that in some global context structure is just lot of extra work that isn't necessary with the profile library compared with other libraries which are focused exclusively on the parsing aspect of things (which is actually pretty trivial). Maybe this doesn't matter for XFS, but other file system utilities may find this issue to be something worthy of consideration when trying to pick a config file library. - Ted ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now! 2018-02-27 3:45 ` Theodore Ts'o @ 2018-02-27 13:43 ` Jan Tulak 2018-02-27 22:14 ` Dave Chinner 0 siblings, 1 reply; 15+ messages in thread From: Jan Tulak @ 2018-02-27 13:43 UTC (permalink / raw) To: Theodore Ts'o Cc: Darrick J. Wong, Luis R. Rodriguez, Eric Sandeen, Dave Chinner, Christoph Hellwig, Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs On Tue, Feb 27, 2018 at 4:45 AM, Theodore Ts'o <tytso@mit.edu> wrote: > One other thing I'll mention is that it was really important to me > that it was very easy to look up values from the config file. So for > me it was a lot more than just the parser. So I can fetch a read a > config parameter from the config file like this: > > profile_get_boolean(ctx->profile, "options", "no_optimize_extents", > 0, 0, &c); > > That's because I use this for a lot more than just file system tuning > parametres, but also for changing how e2fsck / mke2fs behaves. So > trying to parse all of the config file at startup time and having to > stash that in some global context structure is just lot of extra work > that isn't necessary with the profile library compared with other > libraries which are focused exclusively on the parsing aspect of > things (which is actually pretty trivial). > > Maybe this doesn't matter for XFS, but other file system utilities may > find this issue to be something worthy of consideration when trying to > pick a config file library. Neat. But yeah, I don't think the nesting would be useful for xfs. Especially as I prefer the /etc/mkfs.xfs.d/ idea. It might make sense to use something like this, instead of foo.bar style prefixing, though. /etc/mkfs.xfs.d/default: [metadata] crc = 0 [naming] ftype = 0 Cheers, Jan > > - Ted > -- > To unsubscribe from this list: send the line "unsubscribe linux-xfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now! 2018-02-27 13:43 ` Jan Tulak @ 2018-02-27 22:14 ` Dave Chinner 0 siblings, 0 replies; 15+ messages in thread From: Dave Chinner @ 2018-02-27 22:14 UTC (permalink / raw) To: Jan Tulak Cc: Theodore Ts'o, Darrick J. Wong, Luis R. Rodriguez, Eric Sandeen, Christoph Hellwig, Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs On Tue, Feb 27, 2018 at 02:43:44PM +0100, Jan Tulak wrote: > On Tue, Feb 27, 2018 at 4:45 AM, Theodore Ts'o <tytso@mit.edu> wrote: > > One other thing I'll mention is that it was really important to me > > that it was very easy to look up values from the config file. So for > > me it was a lot more than just the parser. So I can fetch a read a > > config parameter from the config file like this: > > > > profile_get_boolean(ctx->profile, "options", "no_optimize_extents", > > 0, 0, &c); That's not the way we've set up mkfs to deal with config file defaults. i.e. the config file creates a "default values" structure that gets used when no CLI parameter was specified. IOWs, all we need the config file to do is to be parsed into a pre-defined structure and the rest of mkfs will already DTRT. i.e. We don't need (or want) mkfs to be directly pulling random values from config files depending on the current option context. > > That's because I use this for a lot more than just file system tuning > > parametres, but also for changing how e2fsck / mke2fs behaves. So > > trying to parse all of the config file at startup time and having to > > stash that in some global context structure is just lot of extra work > > that isn't necessary with the profile library compared with other > > libraries which are focused exclusively on the parsing aspect of > > things (which is actually pretty trivial). > > > > Maybe this doesn't matter for XFS, but other file system utilities may > > find this issue to be something worthy of consideration when trying to > > pick a config file library. > > Neat. But yeah, I don't think the nesting would be useful for xfs. > Especially as I prefer the /etc/mkfs.xfs.d/ idea. It might make sense > to use something like this, instead of foo.bar style prefixing, > though. > > /etc/mkfs.xfs.d/default: > > [metadata] > crc = 0 > > [naming] > ftype = 0 Yup, I like the simplicity of this approach. One config per file, with simple config groups that mirror the CLI grouping and options. Simple for admins, simple for developers, simple to implement and test. We really don't need any more complexity than this, so lets not make it any more complex than it needs to be - that mistake has already been made in recent times with mkfs.xfs modifications, and I don't want to dig us out of that hole again... Cheers, Dave. -- Dave Chinner david@fromorbit.com ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC] mkfs config file bikeshed now! 2018-02-27 3:17 ` Theodore Ts'o 2018-02-27 3:45 ` Theodore Ts'o @ 2018-02-27 18:11 ` Luis R. Rodriguez 1 sibling, 0 replies; 15+ messages in thread From: Luis R. Rodriguez @ 2018-02-27 18:11 UTC (permalink / raw) To: Theodore Ts'o Cc: Darrick J. Wong, Luis R. Rodriguez, Eric Sandeen, Dave Chinner, Christoph Hellwig, Jan Kara, NeilBrown, Jeff Mahoney, linux-xfs On Mon, Feb 26, 2018 at 10:17:57PM -0500, Theodore Ts'o wrote: > So number one, unlike everyone else, I didn't reinvent the wheel. I > grabbed the same wheel I wrote for MIT Kerberos years before. :-) Yes thank you for not re-inventing the wheel. > But hey, to be clear, I'm not the one trying to claim everyone should > use my library (even though it's the best looking and above-average, > ala Lake Wobegon. :-) I started out with libconfig. Clearly that's crap. As I waited for the facelift of mkfs/xfs_mkfs.c I found libini_config and was supportive of it as the best in the community given community traction and good code. As I poked further and extracted e2fsprogs profile library into its own repo and sized it up against libini_config, I'll let it be on me to say: I think its *very* sensible for us to consider using it *provided* we're up to co-maintain a future small shared library in the hopes others will stop also re-inventing the wheel further. Unless I hear otherwise I'll give e2fsprogs profile library a shot for my next iteration under the assumption we can share the library long term. Luis ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2018-02-27 22:14 UTC | newest] Thread overview: 15+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2018-02-26 22:42 [RFC] mkfs config file bikeshed now! Darrick J. Wong 2018-02-26 23:42 ` Eric Sandeen 2018-02-26 23:56 ` Darrick J. Wong 2018-02-27 0:01 ` Eric Sandeen 2018-02-27 0:24 ` Eric Sandeen 2018-02-26 23:43 ` Eric Sandeen 2018-02-27 0:01 ` Luis R. Rodriguez 2018-02-27 0:10 ` Eric Sandeen 2018-02-27 0:15 ` Luis R. Rodriguez 2018-02-27 0:25 ` Darrick J. Wong 2018-02-27 3:17 ` Theodore Ts'o 2018-02-27 3:45 ` Theodore Ts'o 2018-02-27 13:43 ` Jan Tulak 2018-02-27 22:14 ` Dave Chinner 2018-02-27 18:11 ` Luis R. Rodriguez
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).