Re: libata / scsi separation

[Matthew Wilcox <matthew@wil.cx>]

On Tue, Dec 09, 2008 at 04:38:07PM -0600, James Bottomley wrote:
> On Tue, 2008-12-09 at 15:21 -0700, Matthew Wilcox wrote:
> > The performance penalty is certainly measurable.  It's about 1 microsecond
> > per request extra to go from userspace -> scsi -> libata -> driver
> > than it is to go from userspace -> scsi -> driver.  If you issue 400
> > commands per second (as you might do with a 15k RPM SCSI drive), that's
> > 400 microseconds.  If you issue 10,000 commands per second (as you might
> > do with an SSD), that's 10ms of additional CPU time spent in the kernel
> > per second (or 1%).
> 
> Um, not quite.  What you're talking about is increased latency.  It's

Tsk.  I was quite clear I wasn't talking about latency or bandwidth.  I
was talking about the amount of CPU used to keep a device busy.

> not cumulative because we use TCQ (well mostly).  The question is really
> how it impacts the benchmarks, which are mostly throughput based (and
> really, our block layer trades latency for throughput anyway, so it's
> not clear what the impact really is).

If 1% of CPU is being used by the kernel, that's 1% of CPU not available
for the user application (or alternatively an extra centisecond the CPU
could be in a low-power state if you're not CPU-bound).

> > (OK, I haven't measured the overhead of the *SCSI* layer, I've measured
> > the overhead of the *libata* layer.  I think the point here is that you
> > can't measure the difference at a macro level unless you're sending a
> > lot of commands.)
> 
> Perhaps one of the things we should agree on is exactly how we want to
> measure things like this.  Making the layering thinner for less latency
> is usually good ... unless there are other tradeoffs.  I think not
> forcing ata disks to go through SCSI will probably be tradeoff free, but
> we need to make sure it is.

That would certainly be a good idea.  I don't think we have a consensus
about what we should be measuring yet ;-)

-- 
Matthew Wilcox				Intel Open Source Technology Centre
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."