Re: Direct io on block device has performance regression on 2.6.x kernel

[Andrew Morton <akpm@osdl.org>]

"Chen, Kenneth W" <kenneth.w.chen@intel.com> wrote:
>
> Andrew Morton wrote on Wednesday, March 09, 2005 2:45 PM
>  > >
>  > > > Did you generate a kernel profile?
>  > >
>  > >  Top 40 kernel hot functions, percentage is normalized to kernel utilization.
>  > >
>  > >  _spin_unlock_irqrestore		23.54%
>  > >  _spin_unlock_irq			19.27%
>  >
>  > Cripes.
>  >
>  > Is that with CONFIG_PREEMPT?  If so, and if you disable CONFIG_PREEMPT,
>  > this cost should be accounting the the spin_unlock() caller and we can see
>  > who the culprit is.   Perhaps dio->bio_lock.
> 
>  CONFIG_PREEMPT is off.
> 
>  Sorry for all the confusion, I probably shouldn't post the first profile
>  to confuse people.  See 2nd profile that I posted earlier (copied here again).
> 
>  scsi_request_fn		7.54%
>  finish_task_switch	6.25%
>  __blockdev_direct_IO	4.97%
>  __make_request		3.87%
>  scsi_end_request		3.54%
>  dio_bio_end_io		2.70%
>  follow_hugetlb_page	2.39%
>  __wake_up			2.37%
>  aio_complete		1.82%

What are these percentages?  Total CPU time?  The direct-io stuff doesn't
look too bad.  It's surprising that tweaking the direct-io submission code
makes much difference.

hm.  __blockdev_direct_IO() doesn't actually do much.  I assume your damn
compiler went and inlined direct_io_worker() on us.