Hotplug Test Cases or Test Procedures

Hotplug Test Cases or Test Procedures

[Rusty Lynch]

I have a couple of people that have been tasked to start testing the
CompactPCI hotplug
implementation for a couple of platforms, and was wondering if there was an
existing test case or test procedure repository for the hotplug project.
From poking
around the web site I didn't stumble across anything, but it wouldn't be the
first time
I missed the obvious.

If there is no such thing, where would people like to see such
contributions?  At the
very least, this would consist of a few text documents that detailed step by
step manual
procedures for a tester to validate the latest bits on some platform.

Would this be something people would like to see in the hotplug CVS
repository?
Maybe I could approach the LTP?

    -rustyl



-------------------------------------------------------
This sf.net email is sponsored by: To learn the basics of securing 
your web site with SSL, click here to get a FREE TRIAL of a Thawte 
Server Certificate: http://www.gothawte.com/rd524.html
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Hotplug Test Cases or Test Procedures

[Greg KH]

On Tue, Nov 19, 2002 at 06:10:29PM -0800, Rusty Lynch wrote:
> I have a couple of people that have been tasked to start testing the
> CompactPCI hotplug implementation for a couple of platforms, and was
> wondering if there was an existing test case or test procedure
> repository for the hotplug project.

Yeah, testers!  :)

No, there aren't any published test procedures that I know of.  I know a
number of different companies have some private ones that they have used
to test their specific drivers, but so far, I have not been able to
convince anyone of them to make them public.

> From poking around the web site I didn't stumble across anything, but
> it wouldn't be the first time I missed the obvious.

Um, don't confuse the linux-hotplug project with the PCI hotplug
project.  They are two different things (but the PCI hotplug code relies
on the linux-hotplug userspace code to load any needed modules, so they
are intertwined a bit).

> If there is no such thing, where would people like to see such
> contributions?  At the very least, this would consist of a few text
> documents that detailed step by step manual procedures for a tester to
> validate the latest bits on some platform.

That would be great, I would love to have this for my use alone in
testing new code out.

> Would this be something people would like to see in the hotplug CVS
> repository?

Um, maybe in the pci hotplug cvs repository, if you really want to use
CVS.  Or we can just put it up on the web site.

> Maybe I could approach the LTP?

Traditionally they have only wanted tests that they can automate, and
unfortunately, pci hotplug tests (and most driver tests) can't be
automated very easily.  But it can't hurt to try asking them :)

thanks,

greg k-h


-------------------------------------------------------
This sf.net email is sponsored by: To learn the basics of securing 
your web site with SSL, click here to get a FREE TRIAL of a Thawte 
Server Certificate: http://www.gothawte.com/rd524.html
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Hotplug Test Cases or Test Procedures

[Rusty Lynch]

----- Original Message ----- 
From: "Greg KH" <greg@kroah.com>
To: "Rusty Lynch" <rusty@linux.co.intel.com>
Cc: <Linux-hotplug-devel@lists.sourceforge.net>
Sent: Tuesday, November 19, 2002 6:47 PM
Subject: Re: Hotplug Test Cases or Test Procedures


> On Tue, Nov 19, 2002 at 06:10:29PM -0800, Rusty Lynch wrote:
> > I have a couple of people that have been tasked to start testing the
> > CompactPCI hotplug implementation for a couple of platforms, and was
> > wondering if there was an existing test case or test procedure
> > repository for the hotplug project.
> 
> Yeah, testers!  :)
> 
> No, there aren't any published test procedures that I know of.  I know a
> number of different companies have some private ones that they have used
> to test their specific drivers, but so far, I have not been able to
> convince anyone of them to make them public.
> 
> > From poking around the web site I didn't stumble across anything, but
> > it wouldn't be the first time I missed the obvious.
> 
> Um, don't confuse the linux-hotplug project with the PCI hotplug
> project.  They are two different things (but the PCI hotplug code relies
> on the linux-hotplug userspace code to load any needed modules, so they
> are intertwined a bit).
> 

Yeap, I'm mixing up the projects.  Although I'm betting that the test
procedures will end up covering both.

> > If there is no such thing, where would people like to see such
> > contributions?  At the very least, this would consist of a few text
> > documents that detailed step by step manual procedures for a tester to
> > validate the latest bits on some platform.
> 
> That would be great, I would love to have this for my use alone in
> testing new code out.
> 
> > Would this be something people would like to see in the hotplug CVS
> > repository?
> 
> Um, maybe in the pci hotplug cvs repository, if you really want to use
> CVS.  Or we can just put it up on the web site.

Writing the procedures as html and making them accessable via the web site
sounds like a good way to start.

> 
> > Maybe I could approach the LTP?
> 
> Traditionally they have only wanted tests that they can automate, and
> unfortunately, pci hotplug tests (and most driver tests) can't be
> automated very easily.  But it can't hurt to try asking them :)

I'll ping the LTP people to see if they are interested, but that shouldn't
slow us down any.

> 
> thanks,
> 
> greg k-h
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by: To learn the basics of securing 
> your web site with SSL, click here to get a FREE TRIAL of a Thawte 
> Server Certificate: http://www.gothawte.com/rd524.html
> _______________________________________________
> Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
> Linux-hotplug-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel


-------------------------------------------------------
This sf.net email is sponsored by: To learn the basics of securing 
your web site with SSL, click here to get a FREE TRIAL of a Thawte 
Server Certificate: http://www.gothawte.com/rd524.html
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Hotplug Test Cases or Test Procedures

[David Brownell]

Hi Rusty,

What would you want the testcases or procedures to cover?  Are there
tools, like 'cardctl eject' for cardbus/pcmcia, that interact with
hotplug events in testable ways?  Particular drivers/hardware to test,
maybe with setup scripts, or just certain core functionality?

I don't know of any particular tests, but it'd be good to have some!

It's easy enough to handle the "do modules X, Y, and Z load" parts,
maybe even use a 'cardctl' analogue to simulate card insert/eject,
and that part is driver-independent.  Stuff like setup scripts is
harder to characterize, as well as driver-specific (except for the
essentials like "did the script get run").


It's not the same, but there is some info about debugging at the sf.net
site (http://linux-hotplug.sourceforge.net/?selectedÞbug) which talks
about applying one test technique when debugging ... it could easily get
used in regression testing:

  - save events
  - play back later
  - do they produce the intended result?

I suspect a "save the events" routine should be part of the standard
hotplug toolset, it'd have more uses than just debug, test, or logging.


>>>Would this be something people would like to see in the hotplug CVS
>>>repository?

If it's really addressing the ways that user mode tools are
alerted through /sbin/hotplug and /etc/hotplug/* support,
I'd hope so!  Certainly everything that's shared with the
other hotpluggable pci busses should be there, and I agree
with you that


>>..   Or we can just put it up on the web site.
> 
> 
> Writing the procedures as html and making them accessable via the web site
> sounds like a good way to start.

Yes.  Some would need to stay that way (where people need to do
things like swap cards around), no matter how much can be automated.

Are you volunteering to maintain those pages?


>>>Maybe I could approach the LTP?
>>
>>Traditionally they have only wanted tests that they can automate, and
>>unfortunately, pci hotplug tests (and most driver tests) can't be
>>automated very easily.  But it can't hurt to try asking them :)
> 
> 
> I'll ping the LTP people to see if they are interested, but that shouldn't
> slow us down any.

See above.  Some kinds of testing _can_ be automated without
needing industrial robotics!

- Dave





-------------------------------------------------------
This sf.net email is sponsored by: To learn the basics of securing 
your web site with SSL, click here to get a FREE TRIAL of a Thawte 
Server Certificate: http://www.gothawte.com/rd524.html
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Hotplug Test Cases or Test Procedures

[Rusty Lynch]

> Hi Rusty,
>
> What would you want the testcases or procedures to cover?  Are there
> tools, like 'cardctl eject' for cardbus/pcmcia, that interact with
> hotplug events in testable ways?  Particular drivers/hardware to test,
> maybe with setup scripts, or just certain core functionality?

We are just starting to scope this work out, so we don't know what
it will entail.  Initially we need to validate the operation of a couple
of cPCI systems ranging from basic functionality of the zt5550 driver
to system wide testing to verify once a given peripheral board is inserted,
all of the hotplug actions (like loading a driver) happen.  Eventually
there are other bladed architectures with hotplug capibilities people
are dreaming up that my group will need to help implement and test on
Linux.

I'm sure a lot of these test will be very specific a given setup (i.e. a
given
platform hotpluging a specific device), but at the very least it would be
nice
to provide a common way for people to upload other test procedures.

This ping is really my attempt at getting my group to work effectivly in
open source.  It is just too easy to go off and implement something
behind close doors only to find out you did it in a way that is not very
useful to the rest of the world.

>
> I don't know of any particular tests, but it'd be good to have some!
>
> It's easy enough to handle the "do modules X, Y, and Z load" parts,
> maybe even use a 'cardctl' analogue to simulate card insert/eject,
> and that part is driver-independent.  Stuff like setup scripts is
> harder to characterize, as well as driver-specific (except for the
> essentials like "did the script get run").
>
>
> It's not the same, but there is some info about debugging at the sf.net
> site (http://linux-hotplug.sourceforge.net/?selectedÞbug) which talks
> about applying one test technique when debugging ... it could easily get
> used in regression testing:
>
>   - save events
>   - play back later
>   - do they produce the intended result?
>
> I suspect a "save the events" routine should be part of the standard
> hotplug toolset, it'd have more uses than just debug, test, or logging.
>

Cool.  Thanks for the pointer.

>
> >>>Would this be something people would like to see in the hotplug CVS
> >>>repository?
>
> If it's really addressing the ways that user mode tools are
> alerted through /sbin/hotplug and /etc/hotplug/* support,
> I'd hope so!  Certainly everything that's shared with the
> other hotpluggable pci busses should be there, and I agree
> with you that
>
>
> >>..   Or we can just put it up on the web site.
> >
> >
> > Writing the procedures as html and making them accessable via the web
site
> > sounds like a good way to start.
>
> Yes.  Some would need to stay that way (where people need to do
> things like swap cards around), no matter how much can be automated.
>
> Are you volunteering to maintain those pages?
>

Yeap.  I can do that.  Actually I will be pulling in Guomin to work on this.
Guomin is just starting to get familiar with hotplug and pci hotplug, so
there will be some ramp up time.

>
> >>>Maybe I could approach the LTP?
> >>
> >>Traditionally they have only wanted tests that they can automate, and
> >>unfortunately, pci hotplug tests (and most driver tests) can't be
> >>automated very easily.  But it can't hurt to try asking them :)
> >
> >
> > I'll ping the LTP people to see if they are interested, but that
shouldn't
> > slow us down any.
>
> See above.  Some kinds of testing _can_ be automated without
> needing industrial robotics!

BTW, I talked with Stephanie Glass of the LTP project, and she believes
that there is a place we can put manual test procedures.  There test will
just not be part of the "run all" test cases.

I'm thinking adding test to LTP could be a long term goal, but we should
know more after something materializes.

>
> - Dave
>
>



-------------------------------------------------------
This sf.net email is sponsored by: To learn the basics of securing 
your web site with SSL, click here to get a FREE TRIAL of a Thawte 
Server Certificate: http://www.gothawte.com/rd524.html
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel