* RE: Prefetch mmap_sem in ia64_do_page_fault()
2006-03-29 6:54 Prefetch mmap_sem in ia64_do_page_fault() Christoph Lameter
@ 2006-03-29 7:11 ` Chen, Kenneth W
2006-03-29 7:16 ` Christoph Lameter
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Chen, Kenneth W @ 2006-03-29 7:11 UTC (permalink / raw)
To: linux-ia64
Christoph Lameter wrote on Tuesday, March 28, 2006 10:55 PM
> Take a hint from an x86_64 optimization by Arjan van de Ven and use it
> for ia64.
>
> ...
>
> Prefetch the mmap_sem, which is critical for the performance of the page fault
> handler. Maybe we can offset the damage done by the kprobes notifier?
>
> Note: mm may be NULL but I guess that is safe. See
>
> + /* mmap_sem is performance critical.... */
> + prefetchw(&mm->mmap_sem);
> +
I would say push that a couple of lines down after checking !mm
if (in_atomic() || !mm)
goto no_context;
If you insist on prefetching mmap_sem at the very beginning of the
function, then use speculative load, it will "nat" right away with
null pointer without generating any exception at all. It is super
fast because of nat page.
- Ken
^ permalink raw reply [flat|nested] 6+ messages in thread* RE: Prefetch mmap_sem in ia64_do_page_fault()
2006-03-29 6:54 Prefetch mmap_sem in ia64_do_page_fault() Christoph Lameter
2006-03-29 7:11 ` Chen, Kenneth W
@ 2006-03-29 7:16 ` Christoph Lameter
2006-03-29 7:31 ` Chen, Kenneth W
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Christoph Lameter @ 2006-03-29 7:16 UTC (permalink / raw)
To: linux-ia64
On Tue, 28 Mar 2006, Chen, Kenneth W wrote:
> > + /* mmap_sem is performance critical.... */
> > + prefetchw(&mm->mmap_sem);
> > +
>
> I would say push that a couple of lines down after checking !mm
>
> if (in_atomic() || !mm)
> goto no_context;
I initially also thought that would be better.
> If you insist on prefetching mmap_sem at the very beginning of the
> function, then use speculative load, it will "nat" right away with
> null pointer without generating any exception at all. It is super
> fast because of nat page.
I saw David Mosberger's patch from last year
http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;hE8f935527372499b714bf4f8e646a68bb0f52e3
and it seems that he implemented fast handling of these NULL cases?
mm = NULL is very rare so I think we can try to prefetch as
early as possible.
^ permalink raw reply [flat|nested] 6+ messages in thread* RE: Prefetch mmap_sem in ia64_do_page_fault()
2006-03-29 6:54 Prefetch mmap_sem in ia64_do_page_fault() Christoph Lameter
2006-03-29 7:11 ` Chen, Kenneth W
2006-03-29 7:16 ` Christoph Lameter
@ 2006-03-29 7:31 ` Chen, Kenneth W
2006-03-29 7:44 ` Chen, Kenneth W
2006-03-30 2:06 ` Christoph Lameter
4 siblings, 0 replies; 6+ messages in thread
From: Chen, Kenneth W @ 2006-03-29 7:31 UTC (permalink / raw)
To: linux-ia64
Christoph Lameter wrote on Tuesday, March 28, 2006 11:16 PM
> > If you insist on prefetching mmap_sem at the very beginning of the
> > function, then use speculative load, it will "nat" right away with
> > null pointer without generating any exception at all. It is super
> > fast because of nat page.
>
> I saw David Mosberger's patch from last year
>
> http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;hE8f935527372499b714bf4f8e646a68bb0f52e3
>
> and it seems that he implemented fast handling of these NULL cases?
I know about that patch and I'm referring in the context of that commit:
ld8.s r1=[r0] - immediate nat generation, no fault
lfetch.fault [r0] - generate nat page consumption because lfetch is not
considered as a speculative access.
The nat page consumption fault handler should be rather quick, though cpu
still have to fault into the handler, muck around with ipsr.ed bit and
then rfi. What I'm saying is there is an even *faster* way of doing
prefetch.
> mm = NULL is very rare so I think we can try to prefetch as
> early as possible.
Yeah, sure. Looks good!
- Ken
^ permalink raw reply [flat|nested] 6+ messages in thread* RE: Prefetch mmap_sem in ia64_do_page_fault()
2006-03-29 6:54 Prefetch mmap_sem in ia64_do_page_fault() Christoph Lameter
` (2 preceding siblings ...)
2006-03-29 7:31 ` Chen, Kenneth W
@ 2006-03-29 7:44 ` Chen, Kenneth W
2006-03-30 2:06 ` Christoph Lameter
4 siblings, 0 replies; 6+ messages in thread
From: Chen, Kenneth W @ 2006-03-29 7:44 UTC (permalink / raw)
To: linux-ia64
Chen, Kenneth W wrote on Tuesday, March 28, 2006 11:32 PM
> Christoph Lameter wrote on Tuesday, March 28, 2006 11:16 PM
> >
> > I saw David Mosberger's patch from last year
> >
> > and it seems that he implemented fast handling of these NULL cases?
>
> I know about that patch and I'm referring in the context of that commit:
>
> ld8.s r1=[r0] - immediate nat generation, no fault
> lfetch.fault [r0] - generate nat page consumption because lfetch is not
> considered as a speculative access.
>
> The nat page consumption fault handler should be rather quick, though cpu
> still have to fault into the handler, muck around with ipsr.ed bit and
> then rfi. What I'm saying is there is an even *faster* way of doing
> prefetch.
One other caveat with this is that if you have a tlb miss on mm->mmap_sem,
it won't prefetch anything because alt_dtlb_miss handler will nuke the
ld8.s or lfetch instruction :-(
- Ken
^ permalink raw reply [flat|nested] 6+ messages in thread* RE: Prefetch mmap_sem in ia64_do_page_fault()
2006-03-29 6:54 Prefetch mmap_sem in ia64_do_page_fault() Christoph Lameter
` (3 preceding siblings ...)
2006-03-29 7:44 ` Chen, Kenneth W
@ 2006-03-30 2:06 ` Christoph Lameter
4 siblings, 0 replies; 6+ messages in thread
From: Christoph Lameter @ 2006-03-30 2:06 UTC (permalink / raw)
To: linux-ia64
On Tue, 28 Mar 2006, Chen, Kenneth W wrote:
> One other caveat with this is that if you have a tlb miss on mm->mmap_sem,
> it won't prefetch anything because alt_dtlb_miss handler will nuke the
> ld8.s or lfetch instruction :-(
Still no harm done. It is likely (especially in contended situations) that
we have the tlb already.
Maybe we could modify the patch to prefetch the tlb somehow?
^ permalink raw reply [flat|nested] 6+ messages in thread