Re: [Lsf-pc] [LSF/MM TOPIC] a few storage topics

[Jan Kara <jack@suse.cz>]

On Thu 19-01-12 16:08:49, Andrea Arcangeli wrote:
> On Thu, Jan 19, 2012 at 10:46:37AM +0100, Jan Kara wrote:
> > So to make this useful the logic won't be trivial. Thirdly, the benefit is
> > questionable anyway (at least for most of realistic workloads) because
> > flusher thread doesn't write the pages all that often - when there are not
> > many pages, we write them out just once every couple of seconds, when we
> > have lots of dirty pages we cycle through all of them so one page is not
> > written that often.
> 
> If you mean migrate as in mm/migrate.c that's also not cheap, it will
> page fault anybody accessing the page, it'll do the page copy, and
> it'll IPI all cpus that had the mm on the TLB, it locks the page too
> and does all sort of checks. But it's true it'll be CPU bound... while
> I understand the current solution is I/O bound.
  Thanks for explanation. You are right that currently we are I/O bound so
migration is probably faster on most HW but as I said earlier, different
things might end up better in different workloads.

> > > Do you have a more detailed description of the workload? Is it theoretically
> > > avoidable?
> >   See https://lkml.org/lkml/2011/10/23/156. Using page migration or copyout
> > would solve the problems of this guy.
> 
> Copying in the I/O layer should be better than page migration,
> 1) copying the page to a I/O kernel buffer won't involve expensive TLB
> IPIs that migration requires, 2) copying the page to a I/O kernel
> buffer won't cause page faults because of migration entries being set,
> 3) migration has to copy too so the cost on the memory bus is the
> same.
>
> So unless I'm missing something page migration and pte/tlb mangling (I
> mean as in mm/migrate.c) is worse in every way than bounce buffering
> at the I/O layer if you notice the page can be modified while it's
> under I/O.
  Well, but the advantage of migration is that you need to do it only if
the page is redirtied while under IO. Copying to I/O buffer would have to
be done for *all* pages because once we submit the bio, we cannot change
anything. So what will be cheaper depends on how often are redirtied pages
under IO. This is rather rare because pages aren't flushed all that often.
So the effect of stable pages in not observable on throughput. But you can
certainly see it on max latency...

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR