add a new config fragment to kernel/configs

add a new config fragment to kernel/configs

[Anders Roxell]

Hi,

I've seen some confusion about what I'm trying to do with all_abi.config (maybe
not the best name, suggestions are more than welcome).  I should of course
reach out before I started to do this to see if it makes sense to do in the
first place, but forgot about it, sorry.

Let me try to explain what the idea with this new fragment is about.  Adding a
fragment called all_abi.config (lets stick with this name throughout this
email) in kernel/configs would be to enable as much as possible. Then we would
be able to run any test suite with one kernel image.

There have been two different paths I've tried:
1. add fragments that is needed for kselftest, LTP and libhugetlbfs to the
all_abi.config to be able to have the same test coverage as we have today in
LKFT.

2. do allmodconfig and boot that, sadly that doesn't boot. I've tried to bisect
the .config file and ended up with a pile of CONFIG_* options that has to be
enabled/disabled so it booted. The idea was to add these options into the
all_abi.config so we could do 'make allmodconfig all_abi.config'. There are
different options that needs to be enabled/disabled between arm64 and x86
(those are the architectures I've tried on so far).

If we don't think its doable to enable everything into one big kernel, maybe we
can split up the fragments into different testing fragments like:
kselftest.config (that should essentially be todays kselftest-merge, but in a
more visible way), ltp.config, libhugetlbfs.config and so on. The downside here
would be that random test-suites need to send a patch with the fragments that
they have to enable for their test suite.

Cheers,
Anders

Re: [kernelci] add a new config fragment to kernel/configs

[Kevin Hilman]

Anders Roxell <anders.roxell@linaro.org> writes:

> I've seen some confusion about what I'm trying to do with all_abi.config (maybe
> not the best name, suggestions are more than welcome).  I should of course
> reach out before I started to do this to see if it makes sense to do in the
> first place, but forgot about it, sorry.
>
> Let me try to explain what the idea with this new fragment is about.  Adding a
> fragment called all_abi.config (lets stick with this name throughout this
> email) in kernel/configs would be to enable as much as possible. Then we would
> be able to run any test suite with one kernel image.

Thanks for clarifing.  To me, that doesn't sound like ABI, and
especially not *all* ABIs.  It's just a set of interfaces used by common
test suites.  IMO, using ABI is misleading, as when people see that,
they think of all the userspace/kernel interfaces.

> There have been two different paths I've tried:
> 1. add fragments that is needed for kselftest, LTP and libhugetlbfs to the
> all_abi.config to be able to have the same test coverage as we have today in
> LKFT.
>
> 2. do allmodconfig and boot that, sadly that doesn't boot. I've tried to bisect
> the .config file and ended up with a pile of CONFIG_* options that has to be
> enabled/disabled so it booted. The idea was to add these options into the
> all_abi.config so we could do 'make allmodconfig all_abi.config'. There are
> different options that needs to be enabled/disabled between arm64 and x86
> (those are the architectures I've tried on so far).

Having this "minimal boot" defconfig sounds like a good thing to have
independently of the test fragments.

> If we don't think its doable to enable everything into one big kernel, maybe we
> can split up the fragments into different testing fragments like:
> kselftest.config (that should essentially be todays kselftest-merge, but in a
> more visible way), ltp.config, libhugetlbfs.config and so on. The downside here
> would be that random test-suites need to send a patch with the fragments that
> they have to enable for their test suite.

IMO, per test-suite fragments make the most sense, and will much easier
to maintain than one big fragment.  It's also more self-documenting, and
test-suite maintainers might be more motivated to keep things up to
date.   With one massive fragment, it would have to be well documented
(and kept that way) to understand why various options are present, and
which test suites required them etc.

Kevin

Re: [kernelci] add a new config fragment to kernel/configs

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 3146 bytes --]

On Fri, Oct 12, 2018 at 02:57:47PM +0200, Kevin Hilman wrote:
> Anders Roxell <anders.roxell@linaro.org> writes:

> > Let me try to explain what the idea with this new fragment is about.  Adding a
> > fragment called all_abi.config (lets stick with this name throughout this
> > email) in kernel/configs would be to enable as much as possible. Then we would
> > be able to run any test suite with one kernel image.

> Thanks for clarifing.  To me, that doesn't sound like ABI, and
> especially not *all* ABIs.  It's just a set of interfaces used by common
> test suites.  IMO, using ABI is misleading, as when people see that,
> they think of all the userspace/kernel interfaces.

Right, so when we were discussing this internally the thinking was that
we could meet the need for a config fragement that enabled interfaces
for testsuites by enabling all the ABIs that the kernel offers.  This is
a useful thing to do anyway I think and it provides a concrete rule for
what to put in there which is easy to follow and avoids debate about
some of the tuning options that people like to twiddle.  There was also
some thought that we could have other config fragments like one to
enable useful and cheap debug features which could be useful separately.

I do agree that if it is just a "stuff people might want for testing"
config then we shouldn't call it all_abi.

> > 2. do allmodconfig and boot that, sadly that doesn't boot. I've tried to bisect
> > the .config file and ended up with a pile of CONFIG_* options that has to be
> > enabled/disabled so it booted. The idea was to add these options into the
> > all_abi.config so we could do 'make allmodconfig all_abi.config'. There are
> > different options that needs to be enabled/disabled between arm64 and x86
> > (those are the architectures I've tried on so far).

> Having this "minimal boot" defconfig sounds like a good thing to have
> independently of the test fragments.

Definitely.

> IMO, per test-suite fragments make the most sense, and will much easier
> to maintain than one big fragment.  It's also more self-documenting, and
> test-suite maintainers might be more motivated to keep things up to
> date.   With one massive fragment, it would have to be well documented
> (and kept that way) to understand why various options are present, and
> which test suites required them etc.

The idea with having this all_abi config which literally only has things
that are ABIs was to have a simple, clear rule about what goes in which
doesn't need input from most testsuite maintainers or particularly
reference any individual testsuites but will do what's needed for most
of them.  If something's an ABI it should be enabled, if it's not an ABI
then it should be in some other fragment.  That sidesteps the issues
with documentation.

Per testsuite configs have the disadvantage that you've got to work with
every single testsuite to add them (or maintain them separately) and
deal with any churn in kernel config requirements as you move between
kernels with new options or whatever somehow.  Some of the testsuites
will doubtless still end up needing them but hopefully most won't.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [kernelci] add a new config fragment to kernel/configs

[Kevin Hilman]

Mark Brown <broonie@kernel.org> writes:

> On Fri, Oct 12, 2018 at 02:57:47PM +0200, Kevin Hilman wrote:
>> Anders Roxell <anders.roxell@linaro.org> writes:
>
>> > Let me try to explain what the idea with this new fragment is about.  Adding a
>> > fragment called all_abi.config (lets stick with this name throughout this
>> > email) in kernel/configs would be to enable as much as possible. Then we would
>> > be able to run any test suite with one kernel image.
>
>> Thanks for clarifing.  To me, that doesn't sound like ABI, and
>> especially not *all* ABIs.  It's just a set of interfaces used by common
>> test suites.  IMO, using ABI is misleading, as when people see that,
>> they think of all the userspace/kernel interfaces.
>
> Right, so when we were discussing this internally the thinking was that
> we could meet the need for a config fragement that enabled interfaces
> for testsuites by enabling all the ABIs that the kernel offers.  This is
> a useful thing to do anyway I think and it provides a concrete rule for
> what to put in there which is easy to follow and avoids debate about
> some of the tuning options that people like to twiddle.  There was also
> some thought that we could have other config fragments like one to
> enable useful and cheap debug features which could be useful separately.
>
> I do agree that if it is just a "stuff people might want for testing"
> config then we shouldn't call it all_abi.
>
>> > 2. do allmodconfig and boot that, sadly that doesn't boot. I've tried to bisect
>> > the .config file and ended up with a pile of CONFIG_* options that has to be
>> > enabled/disabled so it booted. The idea was to add these options into the
>> > all_abi.config so we could do 'make allmodconfig all_abi.config'. There are
>> > different options that needs to be enabled/disabled between arm64 and x86
>> > (those are the architectures I've tried on so far).
>
>> Having this "minimal boot" defconfig sounds like a good thing to have
>> independently of the test fragments.
>
> Definitely.
>
>> IMO, per test-suite fragments make the most sense, and will much easier
>> to maintain than one big fragment.  It's also more self-documenting, and
>> test-suite maintainers might be more motivated to keep things up to
>> date.   With one massive fragment, it would have to be well documented
>> (and kept that way) to understand why various options are present, and
>> which test suites required them etc.
>
> The idea with having this all_abi config which literally only has things
> that are ABIs was to have a simple, clear rule about what goes in which
> doesn't need input from most testsuite maintainers or particularly
> reference any individual testsuites but will do what's needed for most
> of them.  If something's an ABI it should be enabled, if it's not an ABI
> then it should be in some other fragment.  That sidesteps the issues
> with documentation.

If it's focused on ABI, that sounds fine to me.   However...

> Per testsuite configs have the disadvantage that you've got to work with
> every single testsuite to add them (or maintain them separately) and
> deal with any churn in kernel config requirements as you move between
> kernels with new options or whatever somehow.  Some of the testsuites
> will doubtless still end up needing them but hopefully most won't.

I don't have any proof of this, but I suspect there will testsuites that
are testing things that are not formally ABI, and we'll end up with
per-testsuite defconfigs anyways, which may need to duplicate stuff in
all_abi.config (or specify they need to be combined with all_abi.config)

Personally, I think we should first try the "allmodconfig +
min_boot.config" approach.

If that works, would we still need all_abi.config?

Kevin

Re: [kernelci] add a new config fragment to kernel/configs

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1336 bytes --]

On Mon, Oct 15, 2018 at 03:01:34PM +0200, Kevin Hilman wrote:
> Mark Brown <broonie@kernel.org> writes:

> > Per testsuite configs have the disadvantage that you've got to work with
> > every single testsuite to add them (or maintain them separately) and
> > deal with any churn in kernel config requirements as you move between
> > kernels with new options or whatever somehow.  Some of the testsuites
> > will doubtless still end up needing them but hopefully most won't.

> I don't have any proof of this, but I suspect there will testsuites that
> are testing things that are not formally ABI, and we'll end up with
> per-testsuite defconfigs anyways, which may need to duplicate stuff in
> all_abi.config (or specify they need to be combined with all_abi.config)

Oh, there definitely are.

> Personally, I think we should first try the "allmodconfig +
> min_boot.config" approach.

> If that works, would we still need all_abi.config?

Possibly not, the concerns I guess would be size of the image (though we
could probably arrange to not actually ship all the modules to all the
boards if it was getting in the way) and if all the random debug options
that allmodconfig turns on might be too disruptive (I'm not sure if we'd
be able to turn them off with just a straight min_boot fragment, though
you could extend merge_config).

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [kernelci] add a new config fragment to kernel/configs

[Nicolas Dechesne]

On Mon, Oct 15, 2018 at 3:29 PM Mark Brown <broonie@kernel.org> wrote:
>
> On Mon, Oct 15, 2018 at 03:01:34PM +0200, Kevin Hilman wrote:
> > Mark Brown <broonie@kernel.org> writes:
>
> > > Per testsuite configs have the disadvantage that you've got to work with
> > > every single testsuite to add them (or maintain them separately) and
> > > deal with any churn in kernel config requirements as you move between
> > > kernels with new options or whatever somehow.  Some of the testsuites
> > > will doubtless still end up needing them but hopefully most won't.
>
> > I don't have any proof of this, but I suspect there will testsuites that
> > are testing things that are not formally ABI, and we'll end up with
> > per-testsuite defconfigs anyways, which may need to duplicate stuff in
> > all_abi.config (or specify they need to be combined with all_abi.config)
>
> Oh, there definitely are.
>
> > Personally, I think we should first try the "allmodconfig +
> > min_boot.config" approach.
>
> > If that works, would we still need all_abi.config?

how about making a new tree (in kernelci.org?) that contains "test"
config fragments and maintain them outside of linux master?
kernelci.org could decide which fragment to use for which 'build'. If
some fragments are good enough for mainline, then we might end up
trying to upstream them at some point. That would at least ensure that
everything is properly archived and reproducible. We can mimic what
AOSP is doing for their custom config fragments
(https://android.googlesource.com/kernel/configs/+/master).

>
> Possibly not, the concerns I guess would be size of the image (though we
> could probably arrange to not actually ship all the modules to all the
> boards if it was getting in the way) and if all the random debug options
> that allmodconfig turns on might be too disruptive (I'm not sure if we'd
> be able to turn them off with just a straight min_boot fragment, though
> you could extend merge_config).
>
> 
>

Re: [kernelci] add a new config fragment to kernel/configs

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

On Mon, Oct 15, 2018 at 03:40:20PM +0200, Nicolas Dechesne wrote:
> On Mon, Oct 15, 2018 at 3:29 PM Mark Brown <broonie@kernel.org> wrote:
> > On Mon, Oct 15, 2018 at 03:01:34PM +0200, Kevin Hilman wrote:
> > > Mark Brown <broonie@kernel.org> writes:

> > > Personally, I think we should first try the "allmodconfig +
> > > min_boot.config" approach.

> > > If that works, would we still need all_abi.config?

> how about making a new tree (in kernelci.org?) that contains "test"
> config fragments and maintain them outside of linux master?
> kernelci.org could decide which fragment to use for which 'build'. If
> some fragments are good enough for mainline, then we might end up
> trying to upstream them at some point. That would at least ensure that
> everything is properly archived and reproducible. We can mimic what
> AOSP is doing for their custom config fragments
> (https://android.googlesource.com/kernel/configs/+/master).

The problem is more figuring out which config fragments we need rather
than where to host them.  The thinking with pushing it upstream was that
there's been some interest in collaboration with other people doing test
stuff and there's no particular advantage to having a separate repo.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [kernelci] add a new config fragment to kernel/configs

[Guillaume Tucker]

[-- Attachment #1: Type: text/plain, Size: 693 bytes --]

On Mon, Oct 15, 2018 at 3:05 PM Mark Brown <broonie@kernel.org> wrote:

> On Mon, Oct 15, 2018 at 03:40:20PM +0200, Nicolas Dechesne wrote:
> > On Mon, Oct 15, 2018 at 3:29 PM Mark Brown <broonie@kernel.org> wrote:
> > > On Mon, Oct 15, 2018 at 03:01:34PM +0200, Kevin Hilman wrote:
> > > > Mark Brown <broonie@kernel.org> writes:
>
> > > > Personally, I think we should first try the "allmodconfig +
> > > > min_boot.config" approach.
>
> > > > If that works, would we still need all_abi.config?
>

Can we now drop the patch on staging which looks for the
all_abi.config fragment?


https://github.com/kernelci/kernelci-core-staging/commit/7ee5f44800a5b20ae7f3af308e16ae1502a643f2

Guillaume

[-- Attachment #2: Type: text/html, Size: 1421 bytes --]

Re: [kernelci] add a new config fragment to kernel/configs

[Kevin Hilman]

Guillaume Tucker writes:

> On Mon, Oct 15, 2018 at 3:05 PM Mark Brown <broonie@kernel.org> wrote:
>
>> On Mon, Oct 15, 2018 at 03:40:20PM +0200, Nicolas Dechesne wrote:
>> > On Mon, Oct 15, 2018 at 3:29 PM Mark Brown <broonie@kernel.org> wrote:
>> > > On Mon, Oct 15, 2018 at 03:01:34PM +0200, Kevin Hilman wrote:
>> > > > Mark Brown <broonie@kernel.org> writes:
>>
>> > > > Personally, I think we should first try the "allmodconfig +
>> > > > min_boot.config" approach.
>>
>> > > > If that works, would we still need all_abi.config?
>>
>
> Can we now drop the patch on staging which looks for the
> all_abi.config fragment?

Yes please.

Kevin

>
> https://github.com/kernelci/kernelci-core-staging/commit/7ee5f44800a5b20ae7f3af308e16ae1502a643f2
>
> Guillaume

Re: [kernelci] add a new config fragment to kernel/configs

[Anders Roxell]

On Wed, 31 Oct 2018 at 01:10, Kevin Hilman <khilman@baylibre.com> wrote:
>
>
> Guillaume Tucker writes:
>
> > On Mon, Oct 15, 2018 at 3:05 PM Mark Brown <broonie@kernel.org> wrote:
> >
> >> On Mon, Oct 15, 2018 at 03:40:20PM +0200, Nicolas Dechesne wrote:
> >> > On Mon, Oct 15, 2018 at 3:29 PM Mark Brown <broonie@kernel.org> wrote:
> >> > > On Mon, Oct 15, 2018 at 03:01:34PM +0200, Kevin Hilman wrote:
> >> > > > Mark Brown <broonie@kernel.org> writes:
> >>
> >> > > > Personally, I think we should first try the "allmodconfig +
> >> > > > min_boot.config" approach.
> >>
> >> > > > If that works, would we still need all_abi.config?
> >>
> >
> > Can we now drop the patch on staging which looks for the
> > all_abi.config fragment?
>
> Yes please.

Yes, that make sense.

Anders