[LSF/MM TOPIC] Using buildbot for automated testing of CIFS.KO

[LSF/MM TOPIC] Using buildbot for automated testing of CIFS.KO

[ronnie sahlberg]

Hi Folks,

I am Ronnie Sahlberg, CIFS.KO developer at RedHat.
I want to come to LSF/MM to talk CIFS and network protocols with
colleagues in the FS world.

I have an idea for an informal talk I would like to give:
"Automated testing of kernel fs modules using buildbot. Experiences so
far in CIFS land."

This would be a practical presentation showing examples of a system we
implemented recently
for cifs.ko to have a standard set of tests to run to ensure quality
and (hopefully) avoidance
of regressions for the patches that are sent to Linus for the CIFS.KO module.

At this point we are starting to see very promising results on fixing
bugs before they hit Linus tree.
I would like to show and talk about how we set this up and why we did
it this way.

To see what it looks like right now, this is what I think will go as a
pull request when the next merge window opens:
http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/2/builds/137
http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/4/builds/98

We have already started to add these kind of links in the pull
requests we send upstream.


I would like to talk about how this is set up, what our experiences
are so far, has it improved out quality and our confidence that the
patches are good?
An it would be trivial to reuse for other filesystems if you want to use it.

regards
Ronnie Sahlberg

Re: [LSF/MM TOPIC] Using buildbot for automated testing of CIFS.KO

[J. Bruce Fields]

On Wed, Feb 27, 2019 at 05:22:07PM +1000, ronnie sahlberg wrote:
> Hi Folks,
> 
> I am Ronnie Sahlberg, CIFS.KO developer at RedHat.
> I want to come to LSF/MM to talk CIFS and network protocols with
> colleagues in the FS world.
> 
> I have an idea for an informal talk I would like to give:
> "Automated testing of kernel fs modules using buildbot. Experiences so
> far in CIFS land."
> 
> This would be a practical presentation showing examples of a system we
> implemented recently
> for cifs.ko to have a standard set of tests to run to ensure quality
> and (hopefully) avoidance
> of regressions for the patches that are sent to Linus for the CIFS.KO module.
> 
> At this point we are starting to see very promising results on fixing
> bugs before they hit Linus tree.
> I would like to show and talk about how we set this up and why we did
> it this way.
> 
> To see what it looks like right now, this is what I think will go as a
> pull request when the next merge window opens:
> http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/2/builds/137
> http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/4/builds/98

Am I reading right--did one of those tests take twice as long as the
other?  I wonder why?

Just curious.  Always happy to hear about other folks' test setups.

--b.

> 
> We have already started to add these kind of links in the pull
> requests we send upstream.
> 
> 
> I would like to talk about how this is set up, what our experiences
> are so far, has it improved out quality and our confidence that the
> patches are good?
> An it would be trivial to reuse for other filesystems if you want to use it.
> 
> regards
> Ronnie Sahlberg

Re: [LSF/MM TOPIC] Using buildbot for automated testing of CIFS.KO

[Steve French]

On Wed, Feb 27, 2019 at 2:22 PM J. Bruce Fields <bfields@fieldses.org> wrote:
>
> On Wed, Feb 27, 2019 at 05:22:07PM +1000, ronnie sahlberg wrote:
> > Hi Folks,
> >
> > I am Ronnie Sahlberg, CIFS.KO developer at RedHat.
> > I want to come to LSF/MM to talk CIFS and network protocols with
> > colleagues in the FS world.
> >
> > I have an idea for an informal talk I would like to give:
> > "Automated testing of kernel fs modules using buildbot. Experiences so
> > far in CIFS land."
> >
> > This would be a practical presentation showing examples of a system we
> > implemented recently
> > for cifs.ko to have a standard set of tests to run to ensure quality
> > and (hopefully) avoidance
> > of regressions for the patches that are sent to Linus for the CIFS.KO module.
> >
> > At this point we are starting to see very promising results on fixing
> > bugs before they hit Linus tree.
> > I would like to show and talk about how we set this up and why we did
> > it this way.
> >
> > To see what it looks like right now, this is what I think will go as a
> > pull request when the next merge window opens:
> > http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/2/builds/137
> > http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/4/builds/98
>
> Am I reading right--did one of those tests take twice as long as the
> other?  I wonder why?
>
> Just curious.  Always happy to hear about other folks' test setups.

No - these runs were quite different - two distinct groups of tests.
"cifs-testing" group is longer and tests to three types of server
targets, while the "azure" target tests 67 of the xfstests to Azure
server target (albeit with different configurations for some of the
tests).  There is an additional test group for "DFS" (Global Name
Space) and we will probably add more.

A big hole right now is lack of RDMA ("SMBDirect") tests - in part due
to lack of good options for emulated RDMA in virtual machines.


-- 
Thanks,

Steve

Re: [LSF/MM TOPIC] Using buildbot for automated testing of CIFS.KO

[J. Bruce Fields]

On Wed, Feb 27, 2019 at 11:49:35PM -0600, Steve French wrote:
> On Wed, Feb 27, 2019 at 2:22 PM J. Bruce Fields <bfields@fieldses.org> wrote:
> >
> > On Wed, Feb 27, 2019 at 05:22:07PM +1000, ronnie sahlberg wrote:
> > > To see what it looks like right now, this is what I think will go as a
> > > pull request when the next merge window opens:
> > > http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/2/builds/137
> > > http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/4/builds/98
> >
> > Am I reading right--did one of those tests take twice as long as the
> > other?  I wonder why?
> >
> > Just curious.  Always happy to hear about other folks' test setups.
> 
> No - these runs were quite different - two distinct groups of tests.
> "cifs-testing" group is longer and tests to three types of server
> targets, while the "azure" target tests 67 of the xfstests to Azure
> server target (albeit with different configurations for some of the
> tests).

Oh, got it, thanks.

> There is an additional test group for "DFS" (Global Name
> Space) and we will probably add more.
> 
> A big hole right now is lack of RDMA ("SMBDirect") tests - in part due
> to lack of good options for emulated RDMA in virtual machines.

Yes, I'm currently just assuming Chuck and others are covering this for
me....

--b.