Re: [LSF/MM TOPIC] improving storage testing

[Bart Van Assche <bvanassche@acm.org>]

On Thu, 2019-02-14 at 11:55 +0100, Johannes Thumshirn wrote:
> On Wed, Feb 13, 2019 at 01:07:54PM -0500, Theodore Y. Ts'o wrote:
> > Also, there are expectations about minimum versions of bash that can
> > be supported; but there aren't necessarily for other components such
> > as nvme-cli, and I suspect that it is due to the use of a overly new
> > version of nvme-cli from its git tree.  Is that supposed to work, or
> > should I constrain myself to whatever version is being shipped in
> > Fedora or some other reference distribution?  More generally, what is
> > the overall expectations that should be expected?  xfstests has some
> > extremely expansive set of sed scripts to normalize shell script
> > output to make xfstests extremely portable; will patches along similar
> > lines something that we should be doing for blktests?
> 
> I think this is the root cause of the problems you've sent out mails for this
> week. A lot of blktests test need filtering. See [1] as an example.
> 
> [1] https://github.com/osandov/blktests/pull/34

Hi Johannes,

Output of tools like nvme-cli is not an ABI although an ABI is what is
required to make blktests work reliably. One possible approach is to modify
nvme-cli such that it has two output formats: one output format that is
intended for humans and another that is easy to parse by software. I think
we should consider that approach and compare it to using sed scripts.

Bart.