Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Bug-introducing patches

[Mauro Carvalho Chehab <mchehab+samsung@kernel.org>]

Em Fri, 7 Sep 2018 11:13:20 +0200
Daniel Vetter <daniel.vetter@ffwll.ch> escreveu:

> On Fri, Sep 7, 2018 at 6:27 AM, Theodore Y. Ts'o <tytso@mit.edu> wrote:
> > On Fri, Sep 07, 2018 at 01:49:31AM +0000, Sasha Levin via Ksummit-discuss wrote:  

> > There actually is a perverse incentive to having all of the test
> > 'bots, which is that I suspect some people have come to rely on it to
> > catch problems.  I generally run a full set of regression tests before
> > I push an update to git.kernel.org (it only takes about 2 hours, and
> > 12 VM's :-); and by the time we get to the late -rc's I *always* will
> > do a full regression test.  
> 
> This is what imo a well-run subsystem should sound like from a testing
> pov. All the subsystem specific testing should be done before merging.
> Post-merge is only for integration testing and catching the long-tail
> issues that need months/years of machine time to surface.
> 
> Of course this is much harder for anything that needs physical
> hardware, but even for driver subsystems there's lots you can do with
> test-drivers, selftests and a pile of emulation, to at least catch
> bugs in generic code. And for reasonably sized teams like drm/i915
> building a proper CI is a very obvious investement that will pay off.

IMHO, CI would do even a better job for smaller teams, as they won't
have much resources for testing, but the problem here is that those
teams probably lack resources and money to invest on a physical hardware
to setup a CI infra and to buy the myriad of different hardware to
do regression testing.

Also, some devices are harder to test: how would you check if a camera
microphone is working? How to check if the camera captured images
are ok?

Thanks,
Mauro