* [RFC] Flush huge TLB
@ 2011-10-09 12:53 Hillf Danton
2011-10-10 17:17 ` David Daney
0 siblings, 1 reply; 4+ messages in thread
From: Hillf Danton @ 2011-10-09 12:53 UTC (permalink / raw)
To: linux-mips; +Cc: Ralf Baechle, Jayachandran C.
When flushing TLB, if @vma is backed by huge page, huge TLB should be flushed,
due to the fact that huge page is defined to be far from normal page, and the
flushing is shorten a bit.
Any comment is welcome.
Thanks
Signed-off-by: Hillf Danton <dhillf@gmail.com>
---
--- a/arch/mips/mm/tlb-r4k.c Mon May 30 21:17:04 2011
+++ b/arch/mips/mm/tlb-r4k.c Sun Oct 9 20:50:06 2011
@@ -120,22 +120,35 @@ void local_flush_tlb_range(struct vm_are
if (cpu_context(cpu, mm) != 0) {
unsigned long size, flags;
+ int huge = is_vm_hugetlb_page(vma);
ENTER_CRITICAL(flags);
- size = (end - start + (PAGE_SIZE - 1)) >> PAGE_SHIFT;
- size = (size + 1) >> 1;
+ if (huge) {
+ size = (end - start) / HPAGE_SIZE;
+ } else {
+ size = (end - start + (PAGE_SIZE - 1)) >> PAGE_SHIFT;
+ size = (size + 1) >> 1;
+ }
if (size <= current_cpu_data.tlbsize/2) {
int oldpid = read_c0_entryhi();
int newpid = cpu_asid(cpu, mm);
- start &= (PAGE_MASK << 1);
- end += ((PAGE_SIZE << 1) - 1);
- end &= (PAGE_MASK << 1);
+ if (huge) {
+ start &= HPAGE_MASK;
+ end &= HPAGE_MASK;
+ } else {
+ start &= (PAGE_MASK << 1);
+ end += ((PAGE_SIZE << 1) - 1);
+ end &= (PAGE_MASK << 1);
+ }
while (start < end) {
int idx;
write_c0_entryhi(start | newpid);
- start += (PAGE_SIZE << 1);
+ if (huge)
+ start += HPAGE_SIZE;
+ else
+ start += (PAGE_SIZE << 1);
mtc0_tlbw_hazard();
tlb_probe();
tlb_probe_hazard();
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [RFC] Flush huge TLB 2011-10-09 12:53 [RFC] Flush huge TLB Hillf Danton @ 2011-10-10 17:17 ` David Daney 2011-10-14 13:09 ` Hillf Danton 0 siblings, 1 reply; 4+ messages in thread From: David Daney @ 2011-10-10 17:17 UTC (permalink / raw) To: Hillf Danton, Ralf Baechle; +Cc: linux-mips, Jayachandran C. On 10/09/2011 05:53 AM, Hillf Danton wrote: > When flushing TLB, if @vma is backed by huge page, huge TLB should be flushed, > due to the fact that huge page is defined to be far from normal page, and the > flushing is shorten a bit. > > Any comment is welcome. > Note that the current implementation works, but is not optimal. > Thanks > > Signed-off-by: Hillf Danton<dhillf@gmail.com> > --- > > --- a/arch/mips/mm/tlb-r4k.c Mon May 30 21:17:04 2011 > +++ b/arch/mips/mm/tlb-r4k.c Sun Oct 9 20:50:06 2011 > @@ -120,22 +120,35 @@ void local_flush_tlb_range(struct vm_are > > if (cpu_context(cpu, mm) != 0) { > unsigned long size, flags; > + int huge = is_vm_hugetlb_page(vma); > > ENTER_CRITICAL(flags); > - size = (end - start + (PAGE_SIZE - 1))>> PAGE_SHIFT; > - size = (size + 1)>> 1; > + if (huge) { > + size = (end - start) / HPAGE_SIZE; > + } else { > + size = (end - start + (PAGE_SIZE - 1))>> PAGE_SHIFT; > + size = (size + 1)>> 1; > + } Perhaps: if (huge) { start = round_down(start, HPAGE_SIZE); end = round_up(start, HPAGE_SIZE); size = (end - start) >> HPAGE_SHIFT; } else { start = round_down(start, PAGE_SIZE << 1); end = round_up(start, PAGE_SIZE << 1); size = (end - start) >> (PAGE_SHIFT + 1); } . . . > if (size<= current_cpu_data.tlbsize/2) { Has anybody benchmarked this heuristic? I guess it seems reasonable. > int oldpid = read_c0_entryhi(); > int newpid = cpu_asid(cpu, mm); > > - start&= (PAGE_MASK<< 1); > - end += ((PAGE_SIZE<< 1) - 1); > - end&= (PAGE_MASK<< 1); > + if (huge) { > + start&= HPAGE_MASK; > + end&= HPAGE_MASK; > + } else { > + start&= (PAGE_MASK<< 1); > + end += ((PAGE_SIZE<< 1) - 1); > + end&= (PAGE_MASK<< 1); > + } This stuff is done above so is removed. > while (start< end) { > int idx; > > write_c0_entryhi(start | newpid); > - start += (PAGE_SIZE<< 1); > + if (huge) > + start += HPAGE_SIZE; > + else > + start += (PAGE_SIZE<< 1); > mtc0_tlbw_hazard(); > tlb_probe(); > tlb_probe_hazard(); > > If we do something like that, then... Acked-by: David Daney <david.daney@cavium.com> ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [RFC] Flush huge TLB 2011-10-10 17:17 ` David Daney @ 2011-10-14 13:09 ` Hillf Danton 2011-11-16 14:52 ` Ralf Baechle 0 siblings, 1 reply; 4+ messages in thread From: Hillf Danton @ 2011-10-14 13:09 UTC (permalink / raw) To: David Daney; +Cc: Ralf Baechle, linux-mips, Jayachandran C. On Tue, Oct 11, 2011 at 1:17 AM, David Daney <david.daney@cavium.com> wrote: > On 10/09/2011 05:53 AM, Hillf Danton wrote: >> >> When flushing TLB, if @vma is backed by huge page, huge TLB should be >> flushed, >> due to the fact that huge page is defined to be far from normal page, and >> the >> flushing is shorten a bit. >> >> Any comment is welcome. >> > > Note that the current implementation works, but is not optimal. > >> Thanks >> >> Signed-off-by: Hillf Danton<dhillf@gmail.com> >> --- >> >> --- a/arch/mips/mm/tlb-r4k.c Mon May 30 21:17:04 2011 >> +++ b/arch/mips/mm/tlb-r4k.c Sun Oct 9 20:50:06 2011 >> @@ -120,22 +120,35 @@ void local_flush_tlb_range(struct vm_are >> >> if (cpu_context(cpu, mm) != 0) { >> unsigned long size, flags; >> + int huge = is_vm_hugetlb_page(vma); >> >> ENTER_CRITICAL(flags); >> - size = (end - start + (PAGE_SIZE - 1))>> PAGE_SHIFT; >> - size = (size + 1)>> 1; >> + if (huge) { >> + size = (end - start) / HPAGE_SIZE; > >> + } else { >> + size = (end - start + (PAGE_SIZE - 1))>> >> PAGE_SHIFT; >> + size = (size + 1)>> 1; >> + } > > Perhaps: > if (huge) { > start = round_down(start, HPAGE_SIZE); > end = round_up(start, HPAGE_SIZE); > size = (end - start) >> HPAGE_SHIFT; > } else { > start = round_down(start, PAGE_SIZE << 1); > end = round_up(start, PAGE_SIZE << 1); > size = (end - start) >> (PAGE_SHIFT + 1); > } > . > . > . > >> if (size<= current_cpu_data.tlbsize/2) { > > Has anybody benchmarked this heuristic? I guess it seems reasonable. > >> int oldpid = read_c0_entryhi(); >> int newpid = cpu_asid(cpu, mm); >> >> - start&= (PAGE_MASK<< 1); >> - end += ((PAGE_SIZE<< 1) - 1); >> - end&= (PAGE_MASK<< 1); >> + if (huge) { >> + start&= HPAGE_MASK; >> + end&= HPAGE_MASK; >> + } else { >> + start&= (PAGE_MASK<< 1); >> + end += ((PAGE_SIZE<< 1) - 1); >> + end&= (PAGE_MASK<< 1); >> + } > > This stuff is done above so is removed. > > >> while (start< end) { >> int idx; >> >> write_c0_entryhi(start | newpid); >> - start += (PAGE_SIZE<< 1); >> + if (huge) >> + start += HPAGE_SIZE; >> + else >> + start += (PAGE_SIZE<< 1); >> mtc0_tlbw_hazard(); >> tlb_probe(); >> tlb_probe_hazard(); >> >> > > If we do something like that, then... > > Acked-by: David Daney <david.daney@cavium.com> > Thanks David. It is re-prepared as the following. ---------------------------------------------------------------------------- Subject: Flush huge TLB From: Hillf Danton <dhillf@gmail.com> When flushing TLB, if @vma is backed by huge page, we could flush huge TLB, due to that huge page is defined to be far from normal page. Signed-off-by: Hillf Danton <dhillf@gmail.com> Acked-by: David Daney <david.daney@cavium.com> --- --- a/arch/mips/mm/tlb-r4k.c Mon May 30 21:17:04 2011 +++ b/arch/mips/mm/tlb-r4k.c Sun Oct 9 20:50:06 2011 @@ -120,22 +120,30 @@ void local_flush_tlb_range(struct vm_are if (cpu_context(cpu, mm) != 0) { unsigned long size, flags; + int huge = is_vm_hugetlb_page(vma); ENTER_CRITICAL(flags); - size = (end - start + (PAGE_SIZE - 1)) >> PAGE_SHIFT; - size = (size + 1) >> 1; + if (huge) { + start = round_down(start, HPAGE_SIZE); + end = round_up(end, HPAGE_SIZE); + size = (end - start) >> HPAGE_SHIFT; + } else { + start = round_down(start, PAGE_SIZE << 1); + end = round_up(end, PAGE_SIZE << 1); + size = (end - start) >> (PAGE_SHIFT + 1); + } if (size <= current_cpu_data.tlbsize/2) { int oldpid = read_c0_entryhi(); int newpid = cpu_asid(cpu, mm); - start &= (PAGE_MASK << 1); - end += ((PAGE_SIZE << 1) - 1); - end &= (PAGE_MASK << 1); while (start < end) { int idx; write_c0_entryhi(start | newpid); - start += (PAGE_SIZE << 1); + if (huge) + start += HPAGE_SIZE; + else + start += (PAGE_SIZE << 1); mtc0_tlbw_hazard(); tlb_probe(); tlb_probe_hazard(); ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [RFC] Flush huge TLB 2011-10-14 13:09 ` Hillf Danton @ 2011-11-16 14:52 ` Ralf Baechle 0 siblings, 0 replies; 4+ messages in thread From: Ralf Baechle @ 2011-11-16 14:52 UTC (permalink / raw) To: Hillf Danton; +Cc: David Daney, linux-mips, Jayachandran C. On Fri, Oct 14, 2011 at 09:09:37PM +0800, Hillf Danton wrote: > Subject: Flush huge TLB > From: Hillf Danton <dhillf@gmail.com> > > When flushing TLB, if @vma is backed by huge page, we could flush huge TLB, > due to that huge page is defined to be far from normal page. > > Signed-off-by: Hillf Danton <dhillf@gmail.com> > Acked-by: David Daney <david.daney@cavium.com> I assume this 2nd version was actually meant to be applied, not just for RFC so I've queued it for 3.3. But you better remove that RFC from subject and start a fresh mail thread when posting a patch to avoid confusion! Thanks, Ralf ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2011-11-16 14:53 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2011-10-09 12:53 [RFC] Flush huge TLB Hillf Danton 2011-10-10 17:17 ` David Daney 2011-10-14 13:09 ` Hillf Danton 2011-11-16 14:52 ` Ralf Baechle
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox