* Re: [PATCH] net: mv643xx_eth: remove deprecated inet_lro support
From: Eric Dumazet @ 2013-04-11 19:46 UTC (permalink / raw)
To: Sebastian Hesselbarth
Cc: Andrew Lunn, Jason Cooper, linux-kernel, David S. Miller,
Soeren Moch, Paul Mackerras, linux-arm-kernel, Dale Farnsworth,
Ben Hutchings, netdev, linuxppc-dev, Florian Fainelli,
Lennert Buytenhek, Willy Tarreau
In-Reply-To: <1365707488-28819-1-git-send-email-sebastian.hesselbarth@gmail.com>
On Thu, 2013-04-11 at 21:11 +0200, Sebastian Hesselbarth wrote:
> With recent support for GRO, there is no need to keep both LRO and
> GRO. This patch therefore removes the deprecated inet_lro support
> from mv643xx_eth. This is work is based on an experimental patch
> provided by Eric Dumazet and Willy Tarreau.
>
> Signed-off-by: Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
> Based-on-patch-by: Eric Dumazet <eric.dumazet@gmail.com>
> Based-on-patch-by: Willy Tarreau <w@1wt.eu>
> ---
> Note: This patch is based upon recent cleanup patches and GRO support
> patch for mv643xx_eth.
>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: Lennert Buytenhek <buytenh@wantstofly.org>
> Cc: Andrew Lunn <andrew@lunn.ch>
> Cc: Jason Cooper <jason@lakedaemon.net>
> Cc: Florian Fainelli <florian@openwrt.org>
> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Cc: Paul Mackerras <paulus@samba.org>
> Cc: Dale Farnsworth <dale@farnsworth.org>
> Cc: Ben Hutchings <bhutchings@solarflare.com>
> Cc: Soeren Moch <smoch@web.de>
> Cc: Eric Dumazet <eric.dumazet@gmail.com>
> Cc: Willy Tarreau <w@1wt.eu>
> Cc: netdev@vger.kernel.org
> Cc: linux-arm-kernel@lists.infradead.org
> Cc: linuxppc-dev@lists.ozlabs.org
> Cc: linux-kernel@vger.kernel.org
> ---
> drivers/net/ethernet/marvell/mv643xx_eth.c | 97 +---------------------------
> 1 file changed, 3 insertions(+), 94 deletions(-)
Seems fine to me, but you also could remove "select INET_LRO"
from drivers/net/ethernet/marvell/Kconfig
^ permalink raw reply
* Re: [PATCH] net: mv643xx_eth: remove deprecated inet_lro support
From: Sebastian Hesselbarth @ 2013-04-11 19:52 UTC (permalink / raw)
To: Eric Dumazet
Cc: Andrew Lunn, Jason Cooper, linux-kernel, David S. Miller,
Soeren Moch, Paul Mackerras, linux-arm-kernel, Dale Farnsworth,
Ben Hutchings, netdev, linuxppc-dev, Florian Fainelli,
Lennert Buytenhek, Willy Tarreau
In-Reply-To: <1365709617.3887.185.camel@edumazet-glaptop>
On 04/11/2013 09:46 PM, Eric Dumazet wrote:
> On Thu, 2013-04-11 at 21:11 +0200, Sebastian Hesselbarth wrote:
>> With recent support for GRO, there is no need to keep both LRO and
>> GRO. This patch therefore removes the deprecated inet_lro support
>> from mv643xx_eth. This is work is based on an experimental patch
>> provided by Eric Dumazet and Willy Tarreau.
>>
>> Signed-off-by: Sebastian Hesselbarth<sebastian.hesselbarth@gmail.com>
>> Based-on-patch-by: Eric Dumazet<eric.dumazet@gmail.com>
>> Based-on-patch-by: Willy Tarreau<w@1wt.eu>
>> ---
>> Note: This patch is based upon recent cleanup patches and GRO support
>> patch for mv643xx_eth.
>>
>> Cc: "David S. Miller"<davem@davemloft.net>
>> Cc: Lennert Buytenhek<buytenh@wantstofly.org>
>> Cc: Andrew Lunn<andrew@lunn.ch>
>> Cc: Jason Cooper<jason@lakedaemon.net>
>> Cc: Florian Fainelli<florian@openwrt.org>
>> Cc: Benjamin Herrenschmidt<benh@kernel.crashing.org>
>> Cc: Paul Mackerras<paulus@samba.org>
>> Cc: Dale Farnsworth<dale@farnsworth.org>
>> Cc: Ben Hutchings<bhutchings@solarflare.com>
>> Cc: Soeren Moch<smoch@web.de>
>> Cc: Eric Dumazet<eric.dumazet@gmail.com>
>> Cc: Willy Tarreau<w@1wt.eu>
>> Cc: netdev@vger.kernel.org
>> Cc: linux-arm-kernel@lists.infradead.org
>> Cc: linuxppc-dev@lists.ozlabs.org
>> Cc: linux-kernel@vger.kernel.org
>> ---
>> drivers/net/ethernet/marvell/mv643xx_eth.c | 97 +---------------------------
>> 1 file changed, 3 insertions(+), 94 deletions(-)
>
> Seems fine to me, but you also could remove "select INET_LRO"
> from drivers/net/ethernet/marvell/Kconfig
Ok, I will wait for tomorrow to see if there are more objections and
respin a v2.
Sebastian
^ permalink raw reply
* Re: recommended method of netbooting kernel/dtb in u-boot?
From: Ira W. Snyder @ 2013-04-11 19:50 UTC (permalink / raw)
To: Chris Friesen; +Cc: linuxppc-dev
In-Reply-To: <51670344.2090101@genband.com>
On Thu, Apr 11, 2013 at 12:39:00PM -0600, Chris Friesen wrote:
> On 04/11/2013 12:12 PM, Kumar Gala wrote:
> >
> > On Apr 11, 2013, at 10:44 AM, Chris Friesen wrote:
> >
> >>
> >> Hi all,
> >>
> >> We've got a powerpc system that uses u-boot. In our environment on
> >> bootup u-boot does a DHCP to get networking info, then uses TFTP to
> >> get the kernel, which then does DHCP again and NFS-mounts the
> >> initial root filesystem.
> >>
> >> What's the standard practice for this sort of thing when using
> >> device tree blobs? Do most people use multi-file images or do they
> >> TFTP scripts to load and execute separate kernel/dtb files?
> >
> > We've normally just done multiple tftp fetches and one grabs dtb and
> > one grabs kernel.
>
> Do you hardcode the path to the file in the firmware? Or do you upload
> a script that knows the path to the file?
>
> In our case the path to the boot file(s) depends on which slot the card
> being booted has been inserted in. The DHCP server knows what the path
> is, so it can set dhcpd.conf appropriately, but we need to get that
> information to the firmware on the card being booted.
>
Hello Chris,
I use a hardware setup which sounds similar to yours. The DHCP server
controls which file is sent to each card.
I use the FIT image format to combine a kernel, dtb, and initrd in one
package. Then the U-Boot command "dhcp $address" sets up the network and
tftp's the filename sent by the DHCP server. You don't need to invoke
the U-Boot command "tftp" if you only have one image. "dhcp" is enough.
I used the U-Boot doc/uImage.FIT/*.its examples to get started, and
wrote my own custom .its file for my board. I don't use anything other
than the vmlinux.bin.gz provided by the kernel build.
Hope it helps,
Ira
^ permalink raw reply
* Re: recommended method of netbooting kernel/dtb in u-boot?
From: Chris Friesen @ 2013-04-11 19:59 UTC (permalink / raw)
To: Ira W. Snyder; +Cc: linuxppc-dev
In-Reply-To: <20130411195019.GA31025@ovro.caltech.edu>
On 04/11/2013 01:50 PM, Ira W. Snyder wrote:
> I use a hardware setup which sounds similar to yours. The DHCP server
> controls which file is sent to each card.
>
> I use the FIT image format to combine a kernel, dtb, and initrd in one
> package.
>
> <snip>
>
> I used the U-Boot doc/uImage.FIT/*.its examples to get started, and
> wrote my own custom .its file for my board. I don't use anything other
> than the vmlinux.bin.gz provided by the kernel build.
Okay, that's a good data point, thanks.
Chris
^ permalink raw reply
* Re: [PATCH 0/2] net: mv643xx_eth: use managed clk and allocation
From: David Miller @ 2013-04-11 20:20 UTC (permalink / raw)
To: sebastian.hesselbarth
Cc: andrew, jason, sergei.shtylyov, linux-doc, devicetree-discuss,
linux-kernel, rob.herring, paulus, rob, netdev, linuxppc-dev,
florian, buytenh
In-Reply-To: <1365672574-31123-1-git-send-email-sebastian.hesselbarth@gmail.com>
From: Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
Date: Thu, 11 Apr 2013 11:29:32 +0200
> With introduction of common clock framework and the ability to provide
> gated clocks, mv643xx_eth required calls to get and enable these clock
> gates on some platforms. Back then, common clock framework api wasn't
> safe for architectures without support for common clocks. This has
> changed now and there are also managed (devm_) counterparts for clock
> related functions.
>
> The second patch in this series, also converts kzalloc to devm_kzalloc
> where applicable.
>
> Both patches have been sent to the corresponding mailing lists as
> individual patches before. To get the order required to apply them right,
> this patch set combines both patches into one set.
Both applied to net-next, thanks.
^ permalink raw reply
* Re: [PATCH] net: mv643xx_eth: Add GRO support
From: David Miller @ 2013-04-11 20:22 UTC (permalink / raw)
To: sebastian.hesselbarth
Cc: andrew, jason, linux-kernel, smoch, paulus, linux-arm-kernel,
dale, netdev, linuxppc-dev, florian, buytenh
In-Reply-To: <1365684023-9967-1-git-send-email-sebastian.hesselbarth@gmail.com>
From: Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
Date: Thu, 11 Apr 2013 14:40:23 +0200
> This patch adds GRO support to mv643xx_eth by making it invoke
> napi_gro_receive instead of netif_receive_skb.
>
> Signed-off-by: Soeren Moch <smoch@web.de>
> Signed-off-by: Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
Applied.
^ permalink raw reply
* Re: [PATCH 9/9] powerpc: cpufreq: move cpufreq driver to drivers/cpufreq
From: Rafael J. Wysocki @ 2013-04-11 20:33 UTC (permalink / raw)
To: Viresh Kumar
Cc: robin.randhawa, linux-pm, Liviu.Dudau, linux-kernel, cpufreq,
Steve.Bannister, Paul Mackerras, Olof Johansson, Arnd Bergmann,
arvind.chauhan, linuxppc-dev, linaro-kernel, charles.garcia-tobin
In-Reply-To: <CAKohpomHFgMBAHax=jC_LNd8Gay9aWja+64An7QxnNpaPuuSQQ@mail.gmail.com>
On Thursday, April 11, 2013 12:32:50 PM Viresh Kumar wrote:
> On 11 April 2013 12:29, Olof Johansson <olof@lixom.net> wrote:
> > That's not very nice this late in the staging cycle. Give the powerpc
> > guys a little more time than a single day to come back with test results,
> > please.
>
> Yes we are waiting indefinitely for an Ack from powerpc guys.. If we can
> get a Ack soon, we will get it in 3.10 otherwise next cycle.
Well, I'll be traveling from Saturday morning onwards until the end of the
next week and my window for taking stuff into v3.10 closes in 16 hours.
Whatever is not ready (i.e. ACKed in this particular case) before that time,
won't be taken.
Thanks,
Rafael
--
I speak only for myself.
Rafael J. Wysocki, Intel Open Source Technology Center.
^ permalink raw reply
* Re: [PATCH 0/2] enable the coreint for the mpc85xx 64bit boards
From: Scott Wood @ 2013-04-11 20:45 UTC (permalink / raw)
To: Kumar Gala; +Cc: Wood Scott-B07421, Kevin Hao, linuxppc
In-Reply-To: <6DEE94C2-19EB-487D-99B5-58789C9230BF@kernel.crashing.org>
On 04/11/2013 08:21:24 AM, Kumar Gala wrote:
>=20
> On Apr 10, 2013, at 8:32 PM, Kevin Hao wrote:
>=20
> > Hi,
> >
> > With the rework of the lazy EE, it seems that 64bit kernel works =20
> pretty
> > well on mpc85xx 64bit boards with lazy EE enabled. So this patch =20
> series
> > tries to enable the coreint for these boards by default. This passed
> > the ltp test on a t4240qds board and is based on Kumar's next =20
> branch.
> >
> > ---
> > Kevin Hao (2):
> > powerpc/irq: remove the unneeded flag PACA_IRQ_EE_EDGE
> > powerpc/85xx: enable coreint for all the 64bit boards
> >
> > arch/powerpc/include/asm/hw_irq.h | 1 -
> > arch/powerpc/kernel/exceptions-64e.S | 1 -
> > arch/powerpc/kernel/irq.c | 8 --------
> > arch/powerpc/platforms/85xx/p5020_ds.c | 5 -----
> > arch/powerpc/platforms/85xx/p5040_ds.c | 5 -----
> > arch/powerpc/platforms/85xx/t4240_qds.c | 5 -----
> > 6 files changed, 25 deletions(-)
>=20
> Scott,
>=20
> I'd appreciate your review and ack on this change.
ACK
-Scott=
^ permalink raw reply
* Re: [PATCH 1/9] powerpc,kvm: fix imbalance srcu_read_[un]lock()
From: Paul E. McKenney @ 2013-04-11 21:51 UTC (permalink / raw)
To: Paul Mackerras
Cc: Lai Jiangshan, Gleb Natapov, Marcelo Tosatti, Alexander Graf,
kvm-ppc, linux-kernel, kvm, Andrew Morton, linuxppc-dev
In-Reply-To: <20130317212648.GA4259@iris.ozlabs.ibm.com>
On Mon, Mar 18, 2013 at 08:26:48AM +1100, Paul Mackerras wrote:
> On Sat, Mar 16, 2013 at 12:50:49AM +0800, Lai Jiangshan wrote:
> > At the point of up_out label in kvmppc_hv_setup_htab_rma(),
> > srcu read lock is still held.
> >
> > We have to release it before return.
> >
> > Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
> > Cc: Marcelo Tosatti <mtosatti@redhat.com>
> > Cc: Gleb Natapov <gleb@redhat.com>
> > Cc: Alexander Graf <agraf@suse.de>
> > Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> > Cc: Paul Mackerras <paulus@samba.org>
> > Cc: kvm@vger.kernel.org
> > Cc: kvm-ppc@vger.kernel.org
> > ---
> > arch/powerpc/kvm/book3s_hv.c | 2 +-
> > 1 files changed, 1 insertions(+), 1 deletions(-)
> >
> > diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
> > index 80dcc53..c26740e 100644
> > --- a/arch/powerpc/kvm/book3s_hv.c
> > +++ b/arch/powerpc/kvm/book3s_hv.c
> > @@ -1799,7 +1799,7 @@ static int kvmppc_hv_setup_htab_rma(struct kvm_vcpu *vcpu)
> >
> > up_out:
> > up_read(¤t->mm->mmap_sem);
> > - goto out;
> > + goto out_srcu;
>
> Acked-by: Paul Mackerras <paulus@samba.org>
Thank you both, queued for 3.11 (assuming no one has beat me to it).
Thanx, Paul
^ permalink raw reply
* [BUG?] possible bug in user_enable_block_step()
From: Chris Friesen @ 2013-04-11 22:38 UTC (permalink / raw)
To: linuxppc-dev
Hi,
I'm looking at user_enable_block_step() in current kernels and it has
the following:
#ifdef CONFIG_PPC_ADV_DEBUG_REGS
task->thread.dbcr0 &= ~DBCR0_IC;
task->thread.dbcr0 = DBCR0_IDM | DBCR0_BT;
regs->msr |= MSR_DE;
Should it be as follows?
task->thread.dbcr0 |= DBCR0_IDM | DBCR0_BT;
If not, then what's the point of clearing the DBCR0_IC bit in the
previous line?
Chris
--
Chris Friesen
Software Designer
500 Palladium Drive, Suite 2100
Ottawa, Ontario K2N 1C2, Canada
www.genband.com
office:+1.343.883.2717
chris.friesen@genband.com
^ permalink raw reply
* questions on CONFIG_PPC_ADV_DEBUG_REGS, DBCR0_BRT, and DBCR0_ACTIVE_EVENTS
From: Chris Friesen @ 2013-04-11 23:21 UTC (permalink / raw)
To: linuxppc-dev
Hi all,
Sorry for the bunch of emails, I'm working on some new stuff and running
into issues.
For CONFIG_BOOKE it appears that DBCR0_ACTIVE_EVENTS includes DBCR0_ICMP
but not DBCR0_BRT. Is that just because none of the code paths
currently using DBCR0_ACTIVE_EVENTS need to check DBCR0_BT?
Also, in sys_debug_setcontext() why does SIG_DBG_BRANCH_TRACING return
-EINVAL if CONFIG_PPC_ADV_DEBUG_REGS is set? Would it not be possible
to use DBCR0_BRT?
Thanks,
Chris
--
Chris Friesen
Software Designer
500 Palladium Drive, Suite 2100
Ottawa, Ontario K2N 1C2, Canada
www.genband.com
office:+1.343.883.2717
chris.friesen@genband.com
^ permalink raw reply
* Re: [PATCH -V5 17/25] powerpc/THP: Implement transparent hugepages for ppc64
From: David Gibson @ 2013-04-12 0:51 UTC (permalink / raw)
To: Aneesh Kumar K.V; +Cc: linuxppc-dev, paulus, linux-mm
In-Reply-To: <87sj2xms5u.fsf@linux.vnet.ibm.com>
[-- Attachment #1: Type: text/plain, Size: 20883 bytes --]
On Thu, Apr 11, 2013 at 01:10:29PM +0530, Aneesh Kumar K.V wrote:
> David Gibson <dwg@au1.ibm.com> writes:
>
> > On Thu, Apr 04, 2013 at 11:27:55AM +0530, Aneesh Kumar K.V wrote:
> >> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
> >>
> >> We now have pmd entries covering to 16MB range. To implement THP on powerpc,
> >> we double the size of PMD. The second half is used to deposit the pgtable (PTE page).
> >> We also use the depoisted PTE page for tracking the HPTE information. The information
> >> include [ secondary group | 3 bit hidx | valid ]. We use one byte per each HPTE entry.
> >> With 16MB hugepage and 64K HPTE we need 256 entries and with 4K HPTE we need
> >> 4096 entries. Both will fit in a 4K PTE page.
> >>
> >> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> >> ---
> >> arch/powerpc/include/asm/page.h | 2 +-
> >> arch/powerpc/include/asm/pgtable-ppc64-64k.h | 3 +-
> >> arch/powerpc/include/asm/pgtable-ppc64.h | 2 +-
> >> arch/powerpc/include/asm/pgtable.h | 240 ++++++++++++++++++++
> >> arch/powerpc/mm/pgtable.c | 314 ++++++++++++++++++++++++++
> >> arch/powerpc/mm/pgtable_64.c | 13 ++
> >> arch/powerpc/platforms/Kconfig.cputype | 1 +
> >> 7 files changed, 572 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/arch/powerpc/include/asm/page.h b/arch/powerpc/include/asm/page.h
> >> index 38e7ff6..b927447 100644
> >> --- a/arch/powerpc/include/asm/page.h
> >> +++ b/arch/powerpc/include/asm/page.h
> >> @@ -40,7 +40,7 @@
> >> #ifdef CONFIG_HUGETLB_PAGE
> >> extern unsigned int HPAGE_SHIFT;
> >> #else
> >> -#define HPAGE_SHIFT PAGE_SHIFT
> >> +#define HPAGE_SHIFT PMD_SHIFT
> >
> > That looks like it could break everything except the 64k page size
> > 64-bit base.
>
> How about
It seems very dubious to me to have transparent hugepages enabled
without explicit hugepages in the first place.
>
> diff --git a/arch/powerpc/include/asm/page.h b/arch/powerpc/include/asm/page.h
> index b927447..fadb1ce 100644
> --- a/arch/powerpc/include/asm/page.h
> +++ b/arch/powerpc/include/asm/page.h
> @@ -37,10 +37,19 @@
> #define PAGE_SIZE (ASM_CONST(1) << PAGE_SHIFT)
>
> #ifndef __ASSEMBLY__
> -#ifdef CONFIG_HUGETLB_PAGE
> +/*
> + * With hugetlbfs enabled we allow the HPAGE_SHIFT to run time
> + * configurable. But we enable THP only with 16MB hugepage.
> + * With only THP configured, we force hugepage size to 16MB.
> + * This should ensure that all subarchs that doesn't support
> + * THP continue to work fine with HPAGE_SHIFT usage.
> + */
> +#if defined(CONFIG_HUGETLB_PAGE)
> extern unsigned int HPAGE_SHIFT;
> -#else
> +#elif defined(CONFIG_TRANSPARENT_HUGEPAGE)
> #define HPAGE_SHIFT PMD_SHIFT
> +#else
> +#define HPAGE_SHIFT PAGE_SHIFT
> #endif
> #define HPAGE_SIZE ((1UL) << HPAGE_SHIFT)
> #define HPAGE_MASK (~(HPAGE_SIZE - 1))
>
>
> >
> >> #endif
> >> #define HPAGE_SIZE ((1UL) << HPAGE_SHIFT)
> >> #define HPAGE_MASK (~(HPAGE_SIZE - 1))
> >> diff --git a/arch/powerpc/include/asm/pgtable-ppc64-64k.h b/arch/powerpc/include/asm/pgtable-ppc64-64k.h
> >> index 3c529b4..5c5541a 100644
> >> --- a/arch/powerpc/include/asm/pgtable-ppc64-64k.h
> >> +++ b/arch/powerpc/include/asm/pgtable-ppc64-64k.h
> >> @@ -33,7 +33,8 @@
> >> #define PGDIR_MASK (~(PGDIR_SIZE-1))
> >>
> >> /* Bits to mask out from a PMD to get to the PTE page */
> >> -#define PMD_MASKED_BITS 0x1ff
> >> +/* PMDs point to PTE table fragments which are 4K aligned. */
> >> +#define PMD_MASKED_BITS 0xfff
> >> /* Bits to mask out from a PGD/PUD to get to the PMD page */
> >> #define PUD_MASKED_BITS 0x1ff
> >>
> >> diff --git a/arch/powerpc/include/asm/pgtable-ppc64.h b/arch/powerpc/include/asm/pgtable-ppc64.h
> >> index 0182c20..c0747c7 100644
> >> --- a/arch/powerpc/include/asm/pgtable-ppc64.h
> >> +++ b/arch/powerpc/include/asm/pgtable-ppc64.h
> >> @@ -150,7 +150,7 @@
> >> #define pmd_present(pmd) (pmd_val(pmd) != 0)
> >> #define pmd_clear(pmdp) (pmd_val(*(pmdp)) = 0)
> >> #define pmd_page_vaddr(pmd) (pmd_val(pmd) & ~PMD_MASKED_BITS)
> >> -#define pmd_page(pmd) virt_to_page(pmd_page_vaddr(pmd))
> >> +extern struct page *pmd_page(pmd_t pmd);
> >
> > Does unconditionally changing pmd_page() from a macro to an external
> > function have a noticeable performance impact?
>
> I did measure performance impact with THP enabled. Didn't do a micro
> benchmark to measure impact of this change. Any suggestion what test
> results you would like to see ?
>
> >
> >> #define pud_set(pudp, pudval) (pud_val(*(pudp)) = (pudval))
> >> #define pud_none(pud) (!pud_val(pud))
> >> diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/pgtable.h
> >> index 4b52726..9fbe2a7 100644
> >> --- a/arch/powerpc/include/asm/pgtable.h
> >> +++ b/arch/powerpc/include/asm/pgtable.h
> >> @@ -23,7 +23,247 @@ struct mm_struct;
> >> */
> >> #define PTE_PAGE_HIDX_OFFSET (PTRS_PER_PTE * 8)
> >>
> >> +/* A large part matches with pte bits */
> >> +#define PMD_HUGE_PRESENT 0x001 /* software: pte contains a translation */
> >> +#define PMD_HUGE_USER 0x002 /* matches one of the PP bits */
> >> +#define PMD_HUGE_FILE 0x002 /* (!present only) software: pte holds file offset */
> >
> > Can we actually get hugepage PMDs that are in this state?
>
> Currently we can't, but we would start supporting THP for page cache later.
>
> >
> >> +#define PMD_HUGE_EXEC 0x004 /* No execute on POWER4 and newer (we invert) */
> >> +#define PMD_HUGE_SPLITTING 0x008
> >> +#define PMD_HUGE_SAO 0x010 /* strong Access order */
> >> +#define PMD_HUGE_HASHPTE 0x020
> >> +#define PMD_ISHUGE 0x040
> >> +#define PMD_HUGE_DIRTY 0x080 /* C: page changed */
> >> +#define PMD_HUGE_ACCESSED 0x100 /* R: page referenced */
> >> +#define PMD_HUGE_RW 0x200 /* software: user write access allowed */
> >> +#define PMD_HUGE_BUSY 0x800 /* software: PTE & hash are busy */
> >> +#define PMD_HUGE_HPTEFLAGS (PMD_HUGE_BUSY | PMD_HUGE_HASHPTE)
> >> +/*
> >> + * We keep both the pmd and pte rpn shift same, eventhough we use only
> >> + * lower 12 bits for hugepage flags at pmd level
> >
> > Why?
> >
>
> I was trying to keep PTE level access code to as much similar to huge
> PMD level code. hence retained the same RPN SHIFT, since that would work.
>
> >> + */
> >> +#define PMD_HUGE_RPN_SHIFT PTE_RPN_SHIFT
> >> +#define HUGE_PAGE_SIZE (ASM_CONST(1) << 24)
> >> +#define HUGE_PAGE_MASK (~(HUGE_PAGE_SIZE - 1))
> >> +
>
> .....
>
> >> diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
> >> index 214130a..9f33780 100644
> >> --- a/arch/powerpc/mm/pgtable.c
> >> +++ b/arch/powerpc/mm/pgtable.c
> >> @@ -31,6 +31,7 @@
> >> #include <asm/pgalloc.h>
> >> #include <asm/tlbflush.h>
> >> #include <asm/tlb.h>
> >> +#include <asm/machdep.h>
> >>
> >> #include "mmu_decl.h"
> >>
> >> @@ -240,3 +241,316 @@ void assert_pte_locked(struct mm_struct *mm, unsigned long addr)
> >> }
> >> #endif /* CONFIG_DEBUG_VM */
> >>
> >> +
> >> +#ifdef CONFIG_TRANSPARENT_HUGEPAGE
> >> +static pmd_t set_hugepage_access_flags_filter(pmd_t pmd,
> >> + struct vm_area_struct *vma,
> >> + int dirty)
> >> +{
> >> + return pmd;
> >> +}
> >
> > I don't really see why you're splitting out these trivial ...filter()
> > functions, rather than just doing it inline in the (single) caller.
>
>
> No specific reason other than keeping the hugepmd related code similar
> to PTE related access code. This should enable us to easily enable THP
> support for subarchs.
>
> >
> >> +
> >> +/*
> >> + * This is called when relaxing access to a hugepage. It's also called in the page
> >> + * fault path when we don't hit any of the major fault cases, ie, a minor
> >> + * update of _PAGE_ACCESSED, _PAGE_DIRTY, etc... The generic code will have
> >> + * handled those two for us, we additionally deal with missing execute
> >> + * permission here on some processors
> >> + */
> >> +int pmdp_set_access_flags(struct vm_area_struct *vma, unsigned long address,
> >> + pmd_t *pmdp, pmd_t entry, int dirty)
> >> +{
> >> + int changed;
> >> + entry = set_hugepage_access_flags_filter(entry, vma, dirty);
> >> + changed = !pmd_same(*(pmdp), entry);
> >> + if (changed) {
> >> + __pmdp_set_access_flags(pmdp, entry);
> >> + /*
> >> + * Since we are not supporting SW TLB systems, we don't
> >> + * have any thing similar to flush_tlb_page_nohash()
> >> + */
> >> + }
> >> + return changed;
> >> +}
> >> +
> >> +int pmdp_test_and_clear_young(struct vm_area_struct *vma,
> >> + unsigned long address, pmd_t *pmdp)
> >> +{
> >> + return __pmdp_test_and_clear_young(vma->vm_mm, address, pmdp);
> >> +}
> >> +
> >> +/*
> >> + * We currently remove entries from the hashtable regardless of whether
> >> + * the entry was young or dirty. The generic routines only flush if the
> >> + * entry was young or dirty which is not good enough.
> >> + *
> >> + * We should be more intelligent about this but for the moment we override
> >> + * these functions and force a tlb flush unconditionally
> >> + */
> >> +int pmdp_clear_flush_young(struct vm_area_struct *vma,
> >> + unsigned long address, pmd_t *pmdp)
> >> +{
> >> + return __pmdp_test_and_clear_young(vma->vm_mm, address, pmdp);
> >> +}
> >> +
> >> +/*
> >> + * We mark the pmd splitting and invalidate all the hpte
> >> + * entries for this hugepage.
> >> + */
> >> +void pmdp_splitting_flush(struct vm_area_struct *vma,
> >> + unsigned long address, pmd_t *pmdp)
> >> +{
> >> + unsigned long old, tmp;
> >> +
> >> + VM_BUG_ON(address & ~HPAGE_PMD_MASK);
> >> +#ifdef PTE_ATOMIC_UPDATES
> >> +
> >> + __asm__ __volatile__(
> >> + "1: ldarx %0,0,%3\n\
> >> + andi. %1,%0,%6\n\
> >> + bne- 1b \n\
> >> + ori %1,%0,%4 \n\
> >> + stdcx. %1,0,%3 \n\
> >> + bne- 1b"
> >> + : "=&r" (old), "=&r" (tmp), "=m" (*pmdp)
> >> + : "r" (pmdp), "i" (PMD_HUGE_SPLITTING), "m" (*pmdp), "i" (PMD_HUGE_BUSY)
> >> + : "cc" );
> >> +#else
> >> + old = pmd_val(*pmdp);
> >> + *pmdp = __pmd(old | PMD_HUGE_SPLITTING);
> >> +#endif
> >> + /*
> >> + * If we didn't had the splitting flag set, go and flush the
> >> + * HPTE entries and serialize against gup fast.
> >> + */
> >> + if (!(old & PMD_HUGE_SPLITTING)) {
> >> +#ifdef CONFIG_PPC_STD_MMU_64
> >> + /* We need to flush the hpte */
> >> + if (old & PMD_HUGE_HASHPTE)
> >> + hpte_need_hugepage_flush(vma->vm_mm, address, pmdp);
> >> +#endif
> >> + /* need tlb flush only to serialize against gup-fast */
> >> + flush_tlb_range(vma, address, address + HPAGE_PMD_SIZE);
> >> + }
> >> +}
> >> +
> >> +/*
> >> + * We want to put the pgtable in pmd and use pgtable for tracking
> >> + * the base page size hptes
> >> + */
> >> +void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
> >> + pgtable_t pgtable)
> >> +{
> >> + unsigned long *pgtable_slot;
> >> + assert_spin_locked(&mm->page_table_lock);
> >> + /*
> >> + * we store the pgtable in the second half of PMD
> >> + */
> >> + pgtable_slot = pmdp + PTRS_PER_PMD;
> >> + *pgtable_slot = (unsigned long)pgtable;
> >> +}
> >> +
> >> +#define PTE_FRAG_SIZE (2 * PTRS_PER_PTE * sizeof(pte_t))
> >
> > Another example of why this define should be moved to a header.
>
> will drop.
>
> >
> >> +pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp)
> >> +{
> >> + pgtable_t pgtable;
> >> + unsigned long *pgtable_slot;
> >> +
> >> + assert_spin_locked(&mm->page_table_lock);
> >> + pgtable_slot = pmdp + PTRS_PER_PMD;
> >> + pgtable = (pgtable_t) *pgtable_slot;
> >> + /*
> >> + * We store HPTE information in the deposited PTE fragment.
> >> + * zero out the content on withdraw.
> >> + */
> >> + memset(pgtable, 0, PTE_FRAG_SIZE);
> >> + return pgtable;
> >> +}
> >> +
> >> +/*
> >> + * Since we are looking at latest ppc64, we don't need to worry about
> >> + * i/d cache coherency on exec fault
> >> + */
> >> +static pmd_t set_pmd_filter(pmd_t pmd, unsigned long addr)
> >> +{
> >> + pmd = __pmd(pmd_val(pmd) & ~PMD_HUGE_HPTEFLAGS);
> >> + return pmd;
> >> +}
> >> +
> >> +/*
> >> + * We can make it less convoluted than __set_pte_at, because
> >> + * we can ignore lot of hardware here, because this is only for
> >> + * MPSS
> >> + */
> >> +static inline void __set_pmd_at(struct mm_struct *mm, unsigned long addr,
> >> + pmd_t *pmdp, pmd_t pmd, int percpu)
> >> +{
> >> + /*
> >> + * There is nothing in hash page table now, so nothing to
> >> + * invalidate, set_pte_at is used for adding new entry.
> >> + * For updating we should use update_hugepage_pmd()
> >> + */
> >> + *pmdp = pmd;
> >> +}
> >> +
> >> +/*
> >> + * set a new huge pmd. We should not be called for updating
> >> + * an existing pmd entry. That should go via pmd_hugepage_update.
> >> + */
> >> +void set_pmd_at(struct mm_struct *mm, unsigned long addr,
> >> + pmd_t *pmdp, pmd_t pmd)
> >> +{
> >> + /*
> >> + * Note: mm->context.id might not yet have been assigned as
> >> + * this context might not have been activated yet when this
> >> + * is called.
> >> + */
> >> + pmd = set_pmd_filter(pmd, addr);
> >> +
> >> + __set_pmd_at(mm, addr, pmdp, pmd, 0);
> >> +
> >> +}
> >> +
> >> +void pmdp_invalidate(struct vm_area_struct *vma, unsigned long address,
> >> + pmd_t *pmdp)
> >> +{
> >> + pmd_hugepage_update(vma->vm_mm, address, pmdp, PMD_HUGE_PRESENT);
> >> + flush_tlb_range(vma, address, address + HPAGE_PMD_SIZE);
> >> +}
> >> +
> >> +/*
> >> + * A linux hugepage PMD was changed and the corresponding hash table entry
> >> + * neesd to be flushed.
> >> + *
> >> + * The linux hugepage PMD now include the pmd entries followed by the address
> >> + * to the stashed pgtable_t. The stashed pgtable_t contains the hpte bits.
> >> + * [ secondary group | 3 bit hidx | valid ]. We use one byte per each HPTE entry.
> >> + * With 16MB hugepage and 64K HPTE we need 256 entries and with 4K HPTE we need
> >> + * 4096 entries. Both will fit in a 4K pgtable_t.
> >> + */
> >> +void hpte_need_hugepage_flush(struct mm_struct *mm, unsigned long addr,
> >> + pmd_t *pmdp)
> >> +{
> >> + int ssize, i;
> >> + unsigned long s_addr;
> >> + unsigned int psize, valid;
> >> + unsigned char *hpte_slot_array;
> >> + unsigned long hidx, vpn, vsid, hash, shift, slot;
> >> +
> >> + /*
> >> + * Flush all the hptes mapping this hugepage
> >> + */
> >> + s_addr = addr & HUGE_PAGE_MASK;
> >> + /*
> >> + * The hpte hindex are stored in the pgtable whose address is in the
> >> + * second half of the PMD
> >> + */
> >> + hpte_slot_array = *(char **)(pmdp + PTRS_PER_PMD);
> >> +
> >> + /* get the base page size */
> >> + psize = get_slice_psize(mm, s_addr);
> >> + shift = mmu_psize_defs[psize].shift;
> >> +
> >> + for (i = 0; i < HUGE_PAGE_SIZE/(1ul << shift); i++) {
> >
> > HUGE_PAGE_SIZE >> shift would be a simpler way to do this calculation.
>
> Wonder how I missed that
>
>
> >
> >> + /*
> >> + * 8 bits per each hpte entries
> >> + * 000| [ secondary group (one bit) | hidx (3 bits) | valid bit]
> >> + */
> >> + valid = hpte_slot_array[i] & 0x1;
> >> + if (!valid)
> >> + continue;
> >> + hidx = hpte_slot_array[i] >> 1;
> >> +
> >> + /* get the vpn */
> >> + addr = s_addr + (i * (1ul << shift));
> >> + if (!is_kernel_addr(addr)) {
> >> + ssize = user_segment_size(addr);
> >> + vsid = get_vsid(mm->context.id, addr, ssize);
> >> + WARN_ON(vsid == 0);
> >> + } else {
> >> + vsid = get_kernel_vsid(addr, mmu_kernel_ssize);
> >> + ssize = mmu_kernel_ssize;
> >> + }
> >> +
> >> + vpn = hpt_vpn(addr, vsid, ssize);
> >> + hash = hpt_hash(vpn, shift, ssize);
> >> + if (hidx & _PTEIDX_SECONDARY)
> >> + hash = ~hash;
> >> +
> >> + slot = (hash & htab_hash_mask) * HPTES_PER_GROUP;
> >> + slot += hidx & _PTEIDX_GROUP_IX;
> >> + ppc_md.hpte_invalidate(slot, vpn, psize, ssize, 0);
> >> + }
> >> +}
> >> +
> >> +static pmd_t pmd_set_protbits(pmd_t pmd, pgprot_t pgprot)
> >> +{
> >> + unsigned long pmd_prot = 0;
> >> + unsigned long prot = pgprot_val(pgprot);
> >> +
> >> + if (prot & _PAGE_PRESENT)
> >> + pmd_prot |= PMD_HUGE_PRESENT;
> >> + if (prot & _PAGE_USER)
> >> + pmd_prot |= PMD_HUGE_USER;
> >> + if (prot & _PAGE_FILE)
> >> + pmd_prot |= PMD_HUGE_FILE;
> >> + if (prot & _PAGE_EXEC)
> >> + pmd_prot |= PMD_HUGE_EXEC;
> >> + /*
> >> + * _PAGE_COHERENT should always be set
> >> + */
> >> + VM_BUG_ON(!(prot & _PAGE_COHERENT));
> >> +
> >> + if (prot & _PAGE_SAO)
> >> + pmd_prot |= PMD_HUGE_SAO;
> >
> > This looks dubious because _PAGE_SAO is not a single bit. What
> > happens if WRITETHRU or NO_CACHE is set without the other?
>
> yes that should be
> if ((prot & _PAGE_SAO) == _PAGE_SAO )
>
>
> >
> >> + if (prot & _PAGE_DIRTY)
> >> + pmd_prot |= PMD_HUGE_DIRTY;
> >> + if (prot & _PAGE_ACCESSED)
> >> + pmd_prot |= PMD_HUGE_ACCESSED;
> >> + if (prot & _PAGE_RW)
> >> + pmd_prot |= PMD_HUGE_RW;
> >> +
> >> + pmd_val(pmd) |= pmd_prot;
> >> + return pmd;
> >> +}
> >> +
> >> +pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot)
> >> +{
> >> + pmd_t pmd;
> >> +
> >> + pmd_val(pmd) = pfn << PMD_HUGE_RPN_SHIFT;
> >> + pmd_val(pmd) |= PMD_ISHUGE;
> >> + pmd = pmd_set_protbits(pmd, pgprot);
> >> + return pmd;
> >> +}
> >> +
> >> +pmd_t mk_pmd(struct page *page, pgprot_t pgprot)
> >> +{
> >> + return pfn_pmd(page_to_pfn(page), pgprot);
> >> +}
> >> +
> >> +pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot)
> >> +{
> >> + /* FIXME!! why are this bits cleared ? */
> >
> > You really need to answer this question...
>
> will check.
>
> >
> >> + pmd_val(pmd) &= ~(PMD_HUGE_PRESENT |
> >> + PMD_HUGE_RW |
> >> + PMD_HUGE_EXEC);
> >> + pmd = pmd_set_protbits(pmd, newprot);
> >> + return pmd;
> >> +}
> >> +
> >> +/*
> >> + * This is called at the end of handling a user page fault, when the
> >> + * fault has been handled by updating a HUGE PMD entry in the linux page tables.
> >> + * We use it to preload an HPTE into the hash table corresponding to
> >> + * the updated linux HUGE PMD entry.
> >> + */
> >> +void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr,
> >> + pmd_t *pmd)
> >> +{
> >> + /* FIXME!!
> >> + * Will be done in a later patch
> >> + */
> >
> > If you need another patch to make the code in this patch work, they
> > should probably be folded together.
> >
>
> I have that as TODO, we can do a hash_preload for hugepage here. But I
> don't see we doing that for HugeTLB. So I haven't yet done that for
> hugepage. Do you know why we don't do hash_preload for HugeTLB page ?
>
>
> >> +}
> >> +
> >> +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
> >> diff --git a/arch/powerpc/mm/pgtable_64.c b/arch/powerpc/mm/pgtable_64.c
> >> index e79840b..6fc3488 100644
> >> --- a/arch/powerpc/mm/pgtable_64.c
> >> +++ b/arch/powerpc/mm/pgtable_64.c
> >> @@ -338,6 +338,19 @@ EXPORT_SYMBOL(iounmap);
> >> EXPORT_SYMBOL(__iounmap);
> >> EXPORT_SYMBOL(__iounmap_at);
> >>
> >> +/*
> >> + * For hugepage we have pfn in the pmd, we use PMD_HUGE_RPN_SHIFT bits for flags
> >> + * For PTE page, we have a PTE_FRAG_SIZE (4K) aligned virtual address.
> >> + */
> >> +struct page *pmd_page(pmd_t pmd)
> >> +{
> >> +#ifdef CONFIG_TRANSPARENT_HUGEPAGE
> >> + if (pmd_val(pmd) & PMD_ISHUGE)
> >> + return pfn_to_page(pmd_pfn(pmd));
> >> +#endif
> >> + return virt_to_page(pmd_page_vaddr(pmd));
> >> +}
> >> +
> >> #ifdef CONFIG_PPC_64K_PAGES
> >> /*
> >> * we support 16 fragments per PTE page. This is limited by how many
> >> diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype
> >> index 72afd28..90ee19b 100644
> >> --- a/arch/powerpc/platforms/Kconfig.cputype
> >> +++ b/arch/powerpc/platforms/Kconfig.cputype
> >> @@ -71,6 +71,7 @@ config PPC_BOOK3S_64
> >> select PPC_FPU
> >> select PPC_HAVE_PMU_SUPPORT
> >> select SYS_SUPPORTS_HUGETLBFS
> >> + select HAVE_ARCH_TRANSPARENT_HUGEPAGE if PPC_64K_PAGES
> >>
> >> config PPC_BOOK3E_64
> >> bool "Embedded processors"
>
> -aneesh
>
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/linuxppc-dev
>
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply
* Re: [PATCH -V5 21/25] powerpc: Handle hugepage in perf callchain
From: David Gibson @ 2013-04-12 1:34 UTC (permalink / raw)
To: Aneesh Kumar K.V; +Cc: paulus, linuxppc-dev, linux-mm
In-Reply-To: <1365055083-31956-22-git-send-email-aneesh.kumar@linux.vnet.ibm.com>
[-- Attachment #1: Type: text/plain, Size: 2722 bytes --]
On Thu, Apr 04, 2013 at 11:27:59AM +0530, Aneesh Kumar K.V wrote:
> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> ---
> arch/powerpc/perf/callchain.c | 32 +++++++++++++++++++++-----------
> 1 file changed, 21 insertions(+), 11 deletions(-)
>
> diff --git a/arch/powerpc/perf/callchain.c b/arch/powerpc/perf/callchain.c
> index 578cac7..99262ce 100644
> --- a/arch/powerpc/perf/callchain.c
> +++ b/arch/powerpc/perf/callchain.c
> @@ -115,7 +115,7 @@ static int read_user_stack_slow(void __user *ptr, void *ret, int nb)
> {
> pgd_t *pgdir;
> pte_t *ptep, pte;
> - unsigned shift;
> + unsigned shift, hugepage;
> unsigned long addr = (unsigned long) ptr;
> unsigned long offset;
> unsigned long pfn;
> @@ -125,20 +125,30 @@ static int read_user_stack_slow(void __user *ptr, void *ret, int nb)
> if (!pgdir)
> return -EFAULT;
>
> - ptep = find_linux_pte_or_hugepte(pgdir, addr, &shift, NULL);
> + ptep = find_linux_pte_or_hugepte(pgdir, addr, &shift, &hugepage);
So, this patch pretty much demonstrates that your earlier patch adding
the optional hugepage argument and making the existing callers pass
NULL was broken.
Any code which calls this function and doesn't use and handle the
hugepage return value is horribly broken, so permitting the hugepage
parameter to be optional is itself broken.
I think instead you need to have an early patch that replaces
find_linux_pte_or_hugepte with a new, more abstracted interface, so
that code using it will remain correct when hugepage PMDs become
possible.
> if (!shift)
> shift = PAGE_SHIFT;
>
> - /* align address to page boundary */
> - offset = addr & ((1UL << shift) - 1);
> - addr -= offset;
> -
> - if (ptep == NULL)
> - return -EFAULT;
> - pte = *ptep;
> - if (!pte_present(pte) || !(pte_val(pte) & _PAGE_USER))
> + if (!ptep)
> return -EFAULT;
> - pfn = pte_pfn(pte);
> +
> + if (hugepage) {
> + pmd_t pmd = *(pmd_t *)ptep;
> + shift = mmu_psize_defs[MMU_PAGE_16M].shift;
> + offset = addr & ((1UL << shift) - 1);
> +
> + if (!pmd_large(pmd) || !(pmd_val(pmd) & PMD_HUGE_USER))
> + return -EFAULT;
> + pfn = pmd_pfn(pmd);
> + } else {
> + offset = addr & ((1UL << shift) - 1);
> +
> + pte = *ptep;
> + if (!pte_present(pte) || !(pte_val(pte) & _PAGE_USER))
> + return -EFAULT;
> + pfn = pte_pfn(pte);
> + }
> +
> if (!page_is_ram(pfn))
> return -EFAULT;
>
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply
* Re: [PATCH -V5 19/25] powerpc/THP: Differentiate THP PMD entries from HUGETLB PMD entries
From: David Gibson @ 2013-04-12 1:28 UTC (permalink / raw)
To: Aneesh Kumar K.V; +Cc: paulus, linuxppc-dev, linux-mm
In-Reply-To: <1365055083-31956-20-git-send-email-aneesh.kumar@linux.vnet.ibm.com>
[-- Attachment #1: Type: text/plain, Size: 4436 bytes --]
On Thu, Apr 04, 2013 at 11:27:57AM +0530, Aneesh Kumar K.V wrote:
> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>
> HUGETLB clear the top bit of PMD entries and use that to indicate
> a HUGETLB page directory. Since we store pfns in PMDs for THP,
> we would have the top bit cleared by default. Add the top bit mask
> for THP PMD entries and clear that when we are looking for pmd_pfn.
>
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> ---
> arch/powerpc/include/asm/pgtable.h | 16 +++++++++++++---
> arch/powerpc/mm/pgtable.c | 5 ++++-
> arch/powerpc/mm/pgtable_64.c | 2 +-
> 3 files changed, 18 insertions(+), 5 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/pgtable.h
> index 9fbe2a7..9681de4 100644
> --- a/arch/powerpc/include/asm/pgtable.h
> +++ b/arch/powerpc/include/asm/pgtable.h
> @@ -31,7 +31,7 @@ struct mm_struct;
> #define PMD_HUGE_SPLITTING 0x008
> #define PMD_HUGE_SAO 0x010 /* strong Access order */
> #define PMD_HUGE_HASHPTE 0x020
> -#define PMD_ISHUGE 0x040
> +#define _PMD_ISHUGE 0x040
> #define PMD_HUGE_DIRTY 0x080 /* C: page changed */
> #define PMD_HUGE_ACCESSED 0x100 /* R: page referenced */
> #define PMD_HUGE_RW 0x200 /* software: user write access allowed */
> @@ -44,6 +44,14 @@ struct mm_struct;
> #define PMD_HUGE_RPN_SHIFT PTE_RPN_SHIFT
> #define HUGE_PAGE_SIZE (ASM_CONST(1) << 24)
> #define HUGE_PAGE_MASK (~(HUGE_PAGE_SIZE - 1))
> +/*
> + * HugeTLB looks at the top bit of the Linux page table entries to
> + * decide whether it is a huge page directory or not. Mark HUGE
> + * PMD to differentiate
> + */
> +#define PMD_HUGE_NOT_HUGETLB (ASM_CONST(1) << 63)
> +#define PMD_ISHUGE (_PMD_ISHUGE | PMD_HUGE_NOT_HUGETLB)
Having a define which looks like the name of a boolean flag, but is
two bits strikes me as a really bad idea.
This is one of the many confusions that comes with different pagetable
encodings for transparent and non-transparent hugepages.
Hrm. So your original patch was horribly broken in that your hugepage
PMDs didn't have the top bit set, and so would be confused with hugepd
pointers. Now you're patching it up by forcing the top bit to 1 for
hugepage PMDs. Confusing way of going about it.
> +#define PMD_HUGE_PROTBITS (0xfff | PMD_HUGE_NOT_HUGETLB)
>
> #ifndef __ASSEMBLY__
> extern void hpte_need_hugepage_flush(struct mm_struct *mm, unsigned long addr,
> @@ -70,8 +78,9 @@ static inline int pmd_trans_splitting(pmd_t pmd)
>
> static inline int pmd_trans_huge(pmd_t pmd)
> {
> - return pmd_val(pmd) & PMD_ISHUGE;
> + return ((pmd_val(pmd) & PMD_ISHUGE) == PMD_ISHUGE);
> }
> +
> /* We will enable it in the last patch */
> #define has_transparent_hugepage() 0
> #else
> @@ -84,7 +93,8 @@ static inline unsigned long pmd_pfn(pmd_t pmd)
> /*
> * Only called for hugepage pmd
> */
> - return pmd_val(pmd) >> PMD_HUGE_RPN_SHIFT;
> + unsigned long val = pmd_val(pmd) & ~PMD_HUGE_PROTBITS;
> + return val >> PMD_HUGE_RPN_SHIFT;
> }
>
> static inline int pmd_young(pmd_t pmd)
> diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
> index 9f33780..cf3ca8e 100644
> --- a/arch/powerpc/mm/pgtable.c
> +++ b/arch/powerpc/mm/pgtable.c
> @@ -517,7 +517,10 @@ static pmd_t pmd_set_protbits(pmd_t pmd, pgprot_t pgprot)
> pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot)
> {
> pmd_t pmd;
> -
> + /*
> + * We cannot support that many PFNs
> + */
> + VM_BUG_ON(pfn & PMD_HUGE_NOT_HUGETLB);
> pmd_val(pmd) = pfn << PMD_HUGE_RPN_SHIFT;
> pmd_val(pmd) |= PMD_ISHUGE;
> pmd = pmd_set_protbits(pmd, pgprot);
> diff --git a/arch/powerpc/mm/pgtable_64.c b/arch/powerpc/mm/pgtable_64.c
> index 6fc3488..cd53020 100644
> --- a/arch/powerpc/mm/pgtable_64.c
> +++ b/arch/powerpc/mm/pgtable_64.c
> @@ -345,7 +345,7 @@ EXPORT_SYMBOL(__iounmap_at);
> struct page *pmd_page(pmd_t pmd)
> {
> #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> - if (pmd_val(pmd) & PMD_ISHUGE)
> + if ((pmd_val(pmd) & PMD_ISHUGE) == PMD_ISHUGE)
> return pfn_to_page(pmd_pfn(pmd));
> #endif
> return virt_to_page(pmd_page_vaddr(pmd));
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply
* Re: [PATCH -V5 22/25] powerpc/THP: get_user_pages_fast changes
From: David Gibson @ 2013-04-12 1:41 UTC (permalink / raw)
To: Aneesh Kumar K.V; +Cc: paulus, linuxppc-dev, linux-mm
In-Reply-To: <1365055083-31956-23-git-send-email-aneesh.kumar@linux.vnet.ibm.com>
[-- Attachment #1: Type: text/plain, Size: 3791 bytes --]
On Thu, Apr 04, 2013 at 11:28:00AM +0530, Aneesh Kumar K.V wrote:
> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>
> handle large pages for get_user_pages_fast. Also take care of large page splitting.
>
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> ---
> arch/powerpc/mm/gup.c | 84 +++++++++++++++++++++++++++++++++++++++++++++++--
> 1 file changed, 82 insertions(+), 2 deletions(-)
>
> diff --git a/arch/powerpc/mm/gup.c b/arch/powerpc/mm/gup.c
> index d7efdbf..835c1ae 100644
> --- a/arch/powerpc/mm/gup.c
> +++ b/arch/powerpc/mm/gup.c
> @@ -55,6 +55,72 @@ static noinline int gup_pte_range(pmd_t pmd, unsigned long addr,
> return 1;
> }
>
> +#ifdef CONFIG_TRANSPARENT_HUGEPAGE
> +static inline int gup_huge_pmd(pmd_t *pmdp, unsigned long addr,
> + unsigned long end, int write,
> + struct page **pages, int *nr)
> +{
> + int refs;
> + pmd_t pmd;
> + unsigned long mask;
> + struct page *head, *page, *tail;
> +
> + pmd = *pmdp;
> + mask = PMD_HUGE_PRESENT | PMD_HUGE_USER;
> + if (write)
> + mask |= PMD_HUGE_RW;
> +
> + if ((pmd_val(pmd) & mask) != mask)
> + return 0;
> +
> + /* large pages are never "special" */
> + VM_BUG_ON(!pfn_valid(pmd_pfn(pmd)));
> +
> + refs = 0;
> + head = pmd_page(pmd);
> + page = head + ((addr & ~PMD_MASK) >> PAGE_SHIFT);
> + tail = page;
> + do {
> + VM_BUG_ON(compound_head(page) != head);
> + pages[*nr] = page;
> + (*nr)++;
> + page++;
> + refs++;
> + } while (addr += PAGE_SIZE, addr != end);
> +
> + if (!page_cache_add_speculative(head, refs)) {
> + *nr -= refs;
> + return 0;
> + }
> +
> + if (unlikely(pmd_val(pmd) != pmd_val(*pmdp))) {
> + *nr -= refs;
> + while (refs--)
> + put_page(head);
> + return 0;
> + }
> + /*
> + * Any tail page need their mapcount reference taken before we
> + * return.
> + */
> + while (refs--) {
> + if (PageTail(tail))
> + get_huge_page_tail(tail);
> + tail++;
Is it safe to do this accounting this late?
> + }
> +
> + return 1;
> +}
> +#else
> +
> +static inline int gup_huge_pmd(pmd_t *pmdp, unsigned long addr,
> + unsigned long end, int write,
> + struct page **pages, int *nr)
> +{
> + return 1;
Should be a BUG() here, since this should never be called if
!CONFIG_TRANSPARENT_HUGEPAGE.
> +}
> +#endif
> +
> static int gup_pmd_range(pud_t pud, unsigned long addr, unsigned long end,
> int write, struct page **pages, int *nr)
> {
> @@ -66,9 +132,23 @@ static int gup_pmd_range(pud_t pud, unsigned long addr, unsigned long end,
> pmd_t pmd = *pmdp;
>
> next = pmd_addr_end(addr, end);
> - if (pmd_none(pmd))
> + /*
> + * The pmd_trans_splitting() check below explains why
> + * pmdp_splitting_flush has to flush the tlb, to stop
> + * this gup-fast code from running while we set the
> + * splitting bit in the pmd. Returning zero will take
> + * the slow path that will call wait_split_huge_page()
> + * if the pmd is still in splitting state. gup-fast
> + * can't because it has irq disabled and
> + * wait_split_huge_page() would never return as the
> + * tlb flush IPI wouldn't run.
> + */
> + if (pmd_none(pmd) || pmd_trans_splitting(pmd))
> return 0;
> - if (is_hugepd(pmdp)) {
> + if (unlikely(pmd_large(pmd))) {
> + if (!gup_huge_pmd(pmdp, addr, next, write, pages, nr))
> + return 0;
> + } else if (is_hugepd(pmdp)) {
> if (!gup_hugepd((hugepd_t *)pmdp, PMD_SHIFT,
> addr, next, write, pages, nr))
> return 0;
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply
* [RFC PATCH v2 0/4] try secondary hash before BUG in kernel_map_linear_page()
From: Li Zhong @ 2013-04-12 2:16 UTC (permalink / raw)
To: linuxppc-dev; +Cc: paulus, Li Zhong
Hi Michael,
Here is the updated version, could you please help to review it again?
As you suggested, this version didn't copy the code, but splitted
the logic into a helper function, so both kernel_map_linear_page() and
__hash_page_huge() can use.
Also patch #1 moves some unnecessary code out of the repeating loop, so the
splitting is easier. Patch #3 removes the HPTE_V_BOLTED flag in
kernel_map_linear_page(), it seems not needed based on my understanding.
Changes are split into smaller ones, so each one did only one thing.
Thanks, Zhong
Li Zhong (4):
powerpc: Move the setting of rflags out of loop in __hash_page_huge
powerpc: Split the code trying to insert hpte repeatedly as an helper
function
powerpc: Don't bolt the hpte in kernel_map_linear_page()
powerpc: Try to insert the hptes repeatedly in
kernel_map_linear_page()
arch/powerpc/mm/hash_utils_64.c | 45 +++++++++++++++++++++++++++++++---
arch/powerpc/mm/hugetlbpage-hash64.c | 31 +++++------------------
2 files changed, 47 insertions(+), 29 deletions(-)
--
1.7.9.5
^ permalink raw reply
* [RFC PATCH v2 2/4] powerpc: Split the code trying to insert hpte repeatedly as an helper function
From: Li Zhong @ 2013-04-12 2:16 UTC (permalink / raw)
To: linuxppc-dev; +Cc: paulus, Li Zhong
In-Reply-To: <1365733021-28912-1-git-send-email-zhong@linux.vnet.ibm.com>
Move the logical trying to insert hpte in __hash_page_huge() to an helper
function, so it could also be used by others.
Signed-off-by: Li Zhong <zhong@linux.vnet.ibm.com>
---
arch/powerpc/mm/hash_utils_64.c | 35 ++++++++++++++++++++++++++++++++++
arch/powerpc/mm/hugetlbpage-hash64.c | 31 ++++++------------------------
2 files changed, 41 insertions(+), 25 deletions(-)
diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c
index f410c3e..716f42b 100644
--- a/arch/powerpc/mm/hash_utils_64.c
+++ b/arch/powerpc/mm/hash_utils_64.c
@@ -1230,6 +1230,41 @@ void low_hash_fault(struct pt_regs *regs, unsigned long address, int rc)
bad_page_fault(regs, address, SIGBUS);
}
+long hpte_insert_repeating(unsigned long hash, unsigned long vpn,
+ unsigned long pa, unsigned long rflags,
+ int psize, int ssize)
+{
+ unsigned long hpte_group;
+ long slot;
+
+repeat:
+ hpte_group = ((hash & htab_hash_mask) *
+ HPTES_PER_GROUP) & ~0x7UL;
+
+ /* Insert into the hash table, primary slot */
+ slot = ppc_md.hpte_insert(hpte_group, vpn, pa, rflags, 0,
+ psize, ssize);
+
+ /* Primary is full, try the secondary */
+ if (unlikely(slot == -1)) {
+ hpte_group = ((~hash & htab_hash_mask) *
+ HPTES_PER_GROUP) & ~0x7UL;
+ slot = ppc_md.hpte_insert(hpte_group, vpn, pa, rflags,
+ HPTE_V_SECONDARY,
+ psize, ssize);
+ if (slot == -1) {
+ if (mftb() & 0x1)
+ hpte_group = ((hash & htab_hash_mask) *
+ HPTES_PER_GROUP)&~0x7UL;
+
+ ppc_md.hpte_remove(hpte_group);
+ goto repeat;
+ }
+ }
+
+ return slot;
+}
+
#ifdef CONFIG_DEBUG_PAGEALLOC
static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi)
{
diff --git a/arch/powerpc/mm/hugetlbpage-hash64.c b/arch/powerpc/mm/hugetlbpage-hash64.c
index edb4129..bd7d38b 100644
--- a/arch/powerpc/mm/hugetlbpage-hash64.c
+++ b/arch/powerpc/mm/hugetlbpage-hash64.c
@@ -14,6 +14,10 @@
#include <asm/cacheflush.h>
#include <asm/machdep.h>
+extern long hpte_insert_repeating(unsigned long hash, unsigned long vpn,
+ unsigned long pa, unsigned long rlags,
+ int psize, int ssize);
+
int __hash_page_huge(unsigned long ea, unsigned long access, unsigned long vsid,
pte_t *ptep, unsigned long trap, int local, int ssize,
unsigned int shift, unsigned int mmu_psize)
@@ -83,7 +87,6 @@ int __hash_page_huge(unsigned long ea, unsigned long access, unsigned long vsid,
if (likely(!(old_pte & _PAGE_HASHPTE))) {
unsigned long hash = hpt_hash(vpn, shift, ssize);
- unsigned long hpte_group;
pa = pte_pfn(__pte(old_pte)) << PAGE_SHIFT;
@@ -97,30 +100,8 @@ int __hash_page_huge(unsigned long ea, unsigned long access, unsigned long vsid,
rflags |= (new_pte & (_PAGE_WRITETHRU | _PAGE_NO_CACHE |
_PAGE_COHERENT | _PAGE_GUARDED));
-repeat:
- hpte_group = ((hash & htab_hash_mask) *
- HPTES_PER_GROUP) & ~0x7UL;
-
- /* Insert into the hash table, primary slot */
- slot = ppc_md.hpte_insert(hpte_group, vpn, pa, rflags, 0,
- mmu_psize, ssize);
-
- /* Primary is full, try the secondary */
- if (unlikely(slot == -1)) {
- hpte_group = ((~hash & htab_hash_mask) *
- HPTES_PER_GROUP) & ~0x7UL;
- slot = ppc_md.hpte_insert(hpte_group, vpn, pa, rflags,
- HPTE_V_SECONDARY,
- mmu_psize, ssize);
- if (slot == -1) {
- if (mftb() & 0x1)
- hpte_group = ((hash & htab_hash_mask) *
- HPTES_PER_GROUP)&~0x7UL;
-
- ppc_md.hpte_remove(hpte_group);
- goto repeat;
- }
- }
+ slot = hpte_insert_repeating(hash, vpn, pa,
+ rflags, mmu_psize, ssize);
/*
* Hypervisor failure. Restore old pte and return -1
--
1.7.9.5
^ permalink raw reply related
* [RFC PATCH v2 1/4] powerpc: Move the setting of rflags out of loop in __hash_page_huge
From: Li Zhong @ 2013-04-12 2:16 UTC (permalink / raw)
To: linuxppc-dev; +Cc: paulus, Li Zhong
In-Reply-To: <1365733021-28912-1-git-send-email-zhong@linux.vnet.ibm.com>
It seems that rflags don't get changed in the repeating loop, so move
it out of the loop.
Signed-off-by: Li Zhong <zhong@linux.vnet.ibm.com>
---
arch/powerpc/mm/hugetlbpage-hash64.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/mm/hugetlbpage-hash64.c b/arch/powerpc/mm/hugetlbpage-hash64.c
index cecad34..edb4129 100644
--- a/arch/powerpc/mm/hugetlbpage-hash64.c
+++ b/arch/powerpc/mm/hugetlbpage-hash64.c
@@ -87,10 +87,6 @@ int __hash_page_huge(unsigned long ea, unsigned long access, unsigned long vsid,
pa = pte_pfn(__pte(old_pte)) << PAGE_SHIFT;
-repeat:
- hpte_group = ((hash & htab_hash_mask) *
- HPTES_PER_GROUP) & ~0x7UL;
-
/* clear HPTE slot informations in new PTE */
#ifdef CONFIG_PPC_64K_PAGES
new_pte = (new_pte & ~_PAGE_HPTEFLAGS) | _PAGE_HPTE_SUB0;
@@ -101,6 +97,10 @@ repeat:
rflags |= (new_pte & (_PAGE_WRITETHRU | _PAGE_NO_CACHE |
_PAGE_COHERENT | _PAGE_GUARDED));
+repeat:
+ hpte_group = ((hash & htab_hash_mask) *
+ HPTES_PER_GROUP) & ~0x7UL;
+
/* Insert into the hash table, primary slot */
slot = ppc_md.hpte_insert(hpte_group, vpn, pa, rflags, 0,
mmu_psize, ssize);
--
1.7.9.5
^ permalink raw reply related
* [RFC PATCH v2 3/4] powerpc: Don't bolt the hpte in kernel_map_linear_page()
From: Li Zhong @ 2013-04-12 2:16 UTC (permalink / raw)
To: linuxppc-dev; +Cc: paulus, Li Zhong
In-Reply-To: <1365733021-28912-1-git-send-email-zhong@linux.vnet.ibm.com>
It seems that in kernel_unmap_linear_page(), it only checks whether there
is a map in the linear_map_hash_slots array, so seems we don't need bolt
the hpte.
Signed-off-by: Li Zhong <zhong@linux.vnet.ibm.com>
---
arch/powerpc/mm/hash_utils_64.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c
index 716f42b..a7f54f0 100644
--- a/arch/powerpc/mm/hash_utils_64.c
+++ b/arch/powerpc/mm/hash_utils_64.c
@@ -1281,7 +1281,7 @@ static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi)
if (!vsid)
return;
ret = ppc_md.hpte_insert(hpteg, vpn, __pa(vaddr),
- mode, HPTE_V_BOLTED,
+ mode, 0,
mmu_linear_psize, mmu_kernel_ssize);
BUG_ON (ret < 0);
spin_lock(&linear_map_hash_lock);
--
1.7.9.5
^ permalink raw reply related
* [RFC PATCH v2 4/4] powerpc: Try to insert the hptes repeatedly in kernel_map_linear_page()
From: Li Zhong @ 2013-04-12 2:17 UTC (permalink / raw)
To: linuxppc-dev; +Cc: paulus, Li Zhong
In-Reply-To: <1365733021-28912-1-git-send-email-zhong@linux.vnet.ibm.com>
This patch tries to fix following issue when CONFIG_DEBUG_PAGEALLOC
is enabled:
[ 543.075675] ------------[ cut here ]------------
[ 543.075701] kernel BUG at arch/powerpc/mm/hash_utils_64.c:1239!
[ 543.075714] Oops: Exception in kernel mode, sig: 5 [#1]
[ 543.075722] PREEMPT SMP NR_CPUS=16 DEBUG_PAGEALLOC NUMA pSeries
[ 543.075741] Modules linked in: binfmt_misc ehea
[ 543.075759] NIP: c000000000036eb0 LR: c000000000036ea4 CTR: c00000000005a594
[ 543.075771] REGS: c0000000a90832c0 TRAP: 0700 Not tainted (3.8.0-next-20130222)
[ 543.075781] MSR: 8000000000029032 <SF,EE,ME,IR,DR,RI> CR: 22224482 XER: 00000000
[ 543.075816] SOFTE: 0
[ 543.075823] CFAR: c00000000004c200
[ 543.075830] TASK = c0000000e506b750[23934] 'cc1' THREAD: c0000000a9080000 CPU: 1
GPR00: 0000000000000001 c0000000a9083540 c000000000c600a8 ffffffffffffffff
GPR04: 0000000000000050 fffffffffffffffa c0000000a90834e0 00000000004ff594
GPR08: 0000000000000001 0000000000000000 000000009592d4d8 c000000000c86854
GPR12: 0000000000000002 c000000006ead300 0000000000a51000 0000000000000001
GPR16: f000000003354380 ffffffffffffffff ffffffffffffff80 0000000000000000
GPR20: 0000000000000001 c000000000c600a8 0000000000000001 0000000000000001
GPR24: 0000000003354380 c000000000000000 0000000000000000 c000000000b65950
GPR28: 0000002000000000 00000000000cd50e 0000000000bf50d9 c000000000c7c230
[ 543.076005] NIP [c000000000036eb0] .kernel_map_pages+0x1e0/0x3f8
[ 543.076016] LR [c000000000036ea4] .kernel_map_pages+0x1d4/0x3f8
[ 543.076025] Call Trace:
[ 543.076033] [c0000000a9083540] [c000000000036ea4] .kernel_map_pages+0x1d4/0x3f8 (unreliable)
[ 543.076053] [c0000000a9083640] [c000000000167638] .get_page_from_freelist+0x6cc/0x8dc
[ 543.076067] [c0000000a9083800] [c000000000167a48] .__alloc_pages_nodemask+0x200/0x96c
[ 543.076082] [c0000000a90839c0] [c0000000001ade44] .alloc_pages_vma+0x160/0x1e4
[ 543.076098] [c0000000a9083a80] [c00000000018ce04] .handle_pte_fault+0x1b0/0x7e8
[ 543.076113] [c0000000a9083b50] [c00000000018d5a8] .handle_mm_fault+0x16c/0x1a0
[ 543.076129] [c0000000a9083c00] [c0000000007bf1dc] .do_page_fault+0x4d0/0x7a4
[ 543.076144] [c0000000a9083e30] [c0000000000090e8] handle_page_fault+0x10/0x30
[ 543.076155] Instruction dump:
[ 543.076163] 7c630038 78631d88 e80a0000 f8410028 7c0903a6 e91f01de e96a0010 e84a0008
[ 543.076192] 4e800421 e8410028 7c7107b4 7a200fe0 <0b000000> 7f63db78 48785781 60000000
[ 543.076224] ---[ end trace bd5807e8d6ae186b ]---
Signed-off-by: Li Zhong <zhong@linux.vnet.ibm.com>
---
arch/powerpc/mm/hash_utils_64.c | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c
index a7f54f0..4b449a0 100644
--- a/arch/powerpc/mm/hash_utils_64.c
+++ b/arch/powerpc/mm/hash_utils_64.c
@@ -1272,7 +1272,7 @@ static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi)
unsigned long vsid = get_kernel_vsid(vaddr, mmu_kernel_ssize);
unsigned long vpn = hpt_vpn(vaddr, vsid, mmu_kernel_ssize);
unsigned long mode = htab_convert_pte_flags(PAGE_KERNEL);
- int ret;
+ long ret;
hash = hpt_hash(vpn, PAGE_SHIFT, mmu_kernel_ssize);
hpteg = ((hash & htab_hash_mask) * HPTES_PER_GROUP);
@@ -1280,9 +1280,11 @@ static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi)
/* Don't create HPTE entries for bad address */
if (!vsid)
return;
- ret = ppc_md.hpte_insert(hpteg, vpn, __pa(vaddr),
- mode, 0,
- mmu_linear_psize, mmu_kernel_ssize);
+
+ ret = hpte_insert_repeating(hash, vpn, __pa(vaddr),
+ mode, mmu_linear_psize,
+ mmu_kernel_ssize);
+
BUG_ON (ret < 0);
spin_lock(&linear_map_hash_lock);
BUG_ON(linear_map_hash_slots[lmi] & 0x80);
--
1.7.9.5
^ permalink raw reply related
* RE: [PATCH V6] powerpc/85xx: Add machine check handler to fix PCIe erratum on mpc85xx
From: Jia Hongtao-B38951 @ 2013-04-12 3:05 UTC (permalink / raw)
To: Kumar Gala, Wood Scott-B07421
Cc: linuxppc-dev@lists.ozlabs.org, Li Yang-R58472
In-Reply-To: <A11571AB-BF5B-4943-A4DB-62C33D814165@kernel.crashing.org>
> -----Original Message-----
> From: Kumar Gala [mailto:galak@kernel.crashing.org]
> Sent: Thursday, April 11, 2013 9:47 PM
> To: Jia Hongtao-B38951
> Cc: linuxppc-dev@lists.ozlabs.org; Wood Scott-B07421; Li Yang-R58472
> Subject: Re: [PATCH V6] powerpc/85xx: Add machine check handler to fix
> PCIe erratum on mpc85xx
>=20
>=20
> On Apr 11, 2013, at 3:36 AM, Jia Hongtao wrote:
>=20
> > A PCIe erratum of mpc85xx may causes a core hang when a link of PCIe
> > goes down. when the link goes down, Non-posted transactions issued via
> > the ATMU requiring completion result in an instruction stall.
> > At the same time a machine-check exception is generated to the core to
> > allow further processing by the handler. We implements the handler
> > which skips the instruction caused the stall.
> >
> > This patch depends on patch:
> > powerpc/85xx: Add platform_device declaration to fsl_pci.h
> >
> > Signed-off-by: Zhao Chenhui <b35336@freescale.com>
> > Signed-off-by: Li Yang <leoli@freescale.com>
> > Signed-off-by: Liu Shuo <soniccat.liu@gmail.com>
> > Signed-off-by: Jia Hongtao <hongtao.jia@freescale.com>
> > ---
> > V5:
> > * Move OP and XOP defines to a new header file: asm/ppc-disassemble.h
> > * Add X UX BRX variant of load instruction emulation
> > * Remove A variant of load instruction emulation
> >
> > V4:
> > * Fill rd with all-Fs if the skipped instruction is load and emulate
> > the instruction.
> > * Let KVM/QEMU deal with the exception if the machine check comes from
> KVM.
> >
> > arch/powerpc/include/asm/ppc-disassemble.h | 31 +++++++
> > arch/powerpc/kernel/cpu_setup_fsl_booke.S | 2 +-
> > arch/powerpc/kernel/traps.c | 3 +
> > arch/powerpc/sysdev/fsl_pci.c | 140
> +++++++++++++++++++++++++++++
> > arch/powerpc/sysdev/fsl_pci.h | 6 ++
> > 5 files changed, 181 insertions(+), 1 deletion(-) create mode 100644
> > arch/powerpc/include/asm/ppc-disassemble.h
> >
> > diff --git a/arch/powerpc/include/asm/ppc-disassemble.h
> > b/arch/powerpc/include/asm/ppc-disassemble.h
> > new file mode 100644
> > index 0000000..f9782b8
> > --- /dev/null
> > +++ b/arch/powerpc/include/asm/ppc-disassemble.h
> > @@ -0,0 +1,31 @@
> > +/*
> > + * Copyright 2012-2013 Freescale Semiconductor, Inc.
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU General Public License
> > + * as published by the Free Software Foundation; either version
> > + * 2 of the License, or (at your option) any later version.
> > + *
> > + * provides opcode and xopcode images for use by emulating
> > + * instructions
> > + */
> > +#ifndef _ASM_POWERPC_PPC_DISASSEMBLE_H #define
> > +_ASM_POWERPC_PPC_DISASSEMBLE_H
> > +
>=20
> This should really just be in asm/ppc-opcode.h
Hi Kumar and Scott,
This is the different method of describing instructions so I put them
in a new file.
But I agree that a patch to extract these from existing code is more
well-organized.
Is that OK if I extract these definitions from arch/powerpc/kvm/emulate.c
to asm/ppc-opcode.h? Even though These definitions are different.
Scott,
What's your opinion on this?
>=20
> > +#define OP_LWZ 32
> > +#define OP_LWZU 33
> > +#define OP_LBZ 34
> > +#define OP_LBZU 35
> > +#define OP_LHZ 40
> > +#define OP_LHZU 41
> > +
> > +#define OP_31_XOP_LWZX 23
> > +#define OP_31_XOP_LWZUX 55
> > +#define OP_31_XOP_LBZX 87
> > +#define OP_31_XOP_LBZUX 119
> > +#define OP_31_XOP_LHZX 279
> > +#define OP_31_XOP_LHZUX 311
> > +#define OP_31_XOP_LWBRX 534
> > +#define OP_31_XOP_LHBRX 790
> > +
>=20
> Also, submit a patch to extract these from existing code so we stop
> duplicating them everywhere.
>=20
> > +#endif
> > diff --git a/arch/powerpc/kernel/cpu_setup_fsl_booke.S
> > b/arch/powerpc/kernel/cpu_setup_fsl_booke.S
> > index dcd8819..f1bde90 100644
> > --- a/arch/powerpc/kernel/cpu_setup_fsl_booke.S
> > +++ b/arch/powerpc/kernel/cpu_setup_fsl_booke.S
> > @@ -66,7 +66,7 @@ _GLOBAL(__setup_cpu_e500v2)
> > bl __e500_icache_setup
> > bl __e500_dcache_setup
> > bl __setup_e500_ivors
> > -#ifdef CONFIG_FSL_RIO
> > +#if defined(CONFIG_FSL_RIO) || defined(CONFIG_FSL_PCI)
> > /* Ensure that RFXE is set */
> > mfspr r3,SPRN_HID1
> > oris r3,r3,HID1_RFXE@h
> > diff --git a/arch/powerpc/kernel/traps.c b/arch/powerpc/kernel/traps.c
> > index a008cf5..dd275a4 100644
> > --- a/arch/powerpc/kernel/traps.c
> > +++ b/arch/powerpc/kernel/traps.c
> > @@ -59,6 +59,7 @@
> > #include <asm/fadump.h>
> > #include <asm/switch_to.h>
> > #include <asm/debug.h>
> > +#include <sysdev/fsl_pci.h>
> >
> > #if defined(CONFIG_DEBUGGER) || defined(CONFIG_KEXEC) int
> > (*__debugger)(struct pt_regs *regs) __read_mostly; @@ -556,6 +557,8 @@
> > int machine_check_e500(struct pt_regs *regs)
> > if (reason & MCSR_BUS_RBERR) {
> > if (fsl_rio_mcheck_exception(regs))
> > return 1;
> > + if (fsl_pci_mcheck_exception(regs))
> > + return 1;
> > }
> >
> > printk("Machine check in kernel mode.\n"); diff --git
> > a/arch/powerpc/sysdev/fsl_pci.c b/arch/powerpc/sysdev/fsl_pci.c index
> > 682084d..aaa54c5 100644
> > --- a/arch/powerpc/sysdev/fsl_pci.c
> > +++ b/arch/powerpc/sysdev/fsl_pci.c
> > @@ -26,11 +26,15 @@
> > #include <linux/memblock.h>
> > #include <linux/log2.h>
> > #include <linux/slab.h>
> > +#include <linux/uaccess.h>
> >
> > #include <asm/io.h>
> > #include <asm/prom.h>
> > #include <asm/pci-bridge.h>
> > +#include <asm/ppc-pci.h>
>=20
> what are you pulling in from ppc-pci.h?
is_in_pci_mem_space() uses list_for_each_entry(hose, &hose_list, list_node)=
.
host_list defined in asm/ppc-pci.h.
>=20
> > #include <asm/machdep.h>
> > +#include <asm/disassemble.h>
> > +#include <asm/ppc-disassemble.h>
> > #include <sysdev/fsl_soc.h>
> > #include <sysdev/fsl_pci.h>
> >
> > @@ -826,6 +830,142 @@ u64 fsl_pci_immrbar_base(struct pci_controller
> *hose)
> > return 0;
> > }
> >
> > +#ifdef CONFIG_E500
> > +static int mcheck_handle_load(struct pt_regs *regs, u32 inst) {
> > + unsigned int rd, ra, rb, d;
> > +
> > + rd =3D get_rt(inst);
> > + ra =3D get_ra(inst);
> > + rb =3D get_rb(inst);
> > + d =3D get_d(inst);
> > +
> > + switch (get_op(inst)) {
> > + case 31:
>=20
> What about 64-bit loads (LD...)?
This erratum only happened in E500 core chips in which there is no
64-bit loads.
- Hongtao
^ permalink raw reply
* RE: [PATCH V5] powerpc/MPIC: Add get_version API both for internal and external use
From: Jia Hongtao-B38951 @ 2013-04-12 3:25 UTC (permalink / raw)
To: galak@kernel.crashing.org, Wood Scott-B07421
Cc: linuxppc-dev@lists.ozlabs.org, Li Yang-R58472, Jia Hongtao-B38951
In-Reply-To: <1365562375-11920-1-git-send-email-hongtao.jia@freescale.com>
Hi Kumar,
Scott reviewed this patch and MSI errata patch for weeks.
Here is the link for MSI errata patch:
http://patchwork.ozlabs.org/patch/233211/
Could you please have a review and ACK them if they
look good to you too?
Thanks.
- Hongtao
> -----Original Message-----
> From: Jia Hongtao-B38951
> Sent: Wednesday, April 10, 2013 10:53 AM
> To: linuxppc-dev@lists.ozlabs.org; galak@kernel.crashing.org
> Cc: Wood Scott-B07421; Li Yang-R58472; Jia Hongtao-B38951
> Subject: [PATCH V5] powerpc/MPIC: Add get_version API both for internal
> and external use
>=20
> MPIC version is useful information for both mpic_alloc() and mpic_init().
> The patch provide an API to get MPIC version for reusing the code.
> Also, some other IP block may need MPIC version for their own use.
> The API for external use is also provided.
>=20
> Signed-off-by: Jia Hongtao <hongtao.jia@freescale.com>
> Signed-off-by: Li Yang <leoli@freescale.com>
> ---
> V5:
> * add MPIC_FSL check for fsl_mpic_get_version().
>=20
> V4:
> * change the name of function from mpic_get_version() to
> fsl_mpic_get_version().
>=20
> V3:
> * change the name of function from mpic_primary_get_version() to
> fsl_mpic_primary_get_version().
> * return 0 if mpic_primary is null.
>=20
> V2:
> * Using mpic_get_version() to implement mpic_primary_get_version()
>=20
> arch/powerpc/include/asm/mpic.h | 3 +++
> arch/powerpc/sysdev/mpic.c | 32 +++++++++++++++++++++++++-------
> 2 files changed, 28 insertions(+), 7 deletions(-)
>=20
> diff --git a/arch/powerpc/include/asm/mpic.h
> b/arch/powerpc/include/asm/mpic.h index c0f9ef9..ea6bf72 100644
> --- a/arch/powerpc/include/asm/mpic.h
> +++ b/arch/powerpc/include/asm/mpic.h
> @@ -393,6 +393,9 @@ struct mpic
> #define MPIC_REGSET_STANDARD MPIC_REGSET(0) /* Original
> MPIC */
> #define MPIC_REGSET_TSI108 MPIC_REGSET(1) /* Tsi108/109
> PIC */
>=20
> +/* Get the version of primary MPIC */
> +extern u32 fsl_mpic_primary_get_version(void);
> +
> /* Allocate the controller structure and setup the linux irq descs
> * for the range if interrupts passed in. No HW initialization is
> * actually performed.
> diff --git a/arch/powerpc/sysdev/mpic.c b/arch/powerpc/sysdev/mpic.c
> index d30e6a6..47ef4ba 100644
> --- a/arch/powerpc/sysdev/mpic.c
> +++ b/arch/powerpc/sysdev/mpic.c
> @@ -1165,10 +1165,33 @@ static struct irq_domain_ops mpic_host_ops =3D {
> .xlate =3D mpic_host_xlate,
> };
>=20
> +static u32 fsl_mpic_get_version(struct mpic *mpic) {
> + u32 brr1;
> +
> + if (!(mpic->flags & MPIC_FSL))
> + return 0;
> +
> + brr1 =3D _mpic_read(mpic->reg_type, &mpic->thiscpuregs,
> + MPIC_FSL_BRR1);
> +
> + return brr1 & MPIC_FSL_BRR1_VER;
> +}
> +
> /*
> * Exported functions
> */
>=20
> +u32 fsl_mpic_primary_get_version(void)
> +{
> + struct mpic *mpic =3D mpic_primary;
> +
> + if (mpic)
> + return fsl_mpic_get_version(mpic);
> +
> + return 0;
> +}
> +
> struct mpic * __init mpic_alloc(struct device_node *node,
> phys_addr_t phys_addr,
> unsigned int flags,
> @@ -1315,7 +1338,6 @@ struct mpic * __init mpic_alloc(struct device_node
> *node,
> mpic_map(mpic, mpic->paddr, &mpic->tmregs, MPIC_INFO(TIMER_BASE),
> 0x1000);
>=20
> if (mpic->flags & MPIC_FSL) {
> - u32 brr1;
> int ret;
>=20
> /*
> @@ -1326,9 +1348,7 @@ struct mpic * __init mpic_alloc(struct device_node
> *node,
> mpic_map(mpic, mpic->paddr, &mpic->thiscpuregs,
> MPIC_CPU_THISBASE, 0x1000);
>=20
> - brr1 =3D _mpic_read(mpic->reg_type, &mpic->thiscpuregs,
> - MPIC_FSL_BRR1);
> - fsl_version =3D brr1 & MPIC_FSL_BRR1_VER;
> + fsl_version =3D fsl_mpic_get_version(mpic);
>=20
> /* Error interrupt mask register (EIMR) is required for
> * handling individual device error interrupts. EIMR @@ -
> 1518,9 +1538,7 @@ void __init mpic_init(struct mpic *mpic)
> mpic_cpu_write(MPIC_INFO(CPU_CURRENT_TASK_PRI), 0xf);
>=20
> if (mpic->flags & MPIC_FSL) {
> - u32 brr1 =3D _mpic_read(mpic->reg_type, &mpic->thiscpuregs,
> - MPIC_FSL_BRR1);
> - u32 version =3D brr1 & MPIC_FSL_BRR1_VER;
> + u32 version =3D fsl_mpic_get_version(mpic);
>=20
> /*
> * Timer group B is present at the latest in MPIC 3.1 (e.g.
> --
> 1.8.0
^ permalink raw reply
* Re: [PATCH -V5 20/25] powerpc/THP: Add code to handle HPTE faults for large pages
From: David Gibson @ 2013-04-12 4:01 UTC (permalink / raw)
To: Aneesh Kumar K.V; +Cc: paulus, linuxppc-dev, linux-mm
In-Reply-To: <1365055083-31956-21-git-send-email-aneesh.kumar@linux.vnet.ibm.com>
[-- Attachment #1: Type: text/plain, Size: 21539 bytes --]
On Thu, Apr 04, 2013 at 11:27:58AM +0530, Aneesh Kumar K.V wrote:
> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>
> We now have pmd entries covering to 16MB range. To implement THP on powerpc,
> we double the size of PMD. The second half is used to deposit the pgtable (PTE page).
> We also use the depoisted PTE page for tracking the HPTE information. The information
> include [ secondary group | 3 bit hidx | valid ]. We use one byte per each HPTE entry.
> With 16MB hugepage and 64K HPTE we need 256 entries and with 4K HPTE we need
> 4096 entries. Both will fit in a 4K PTE page.
This description is a duplicate of an earlier path. Both are
innaccurate for the patches they are now attached to.
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> ---
> arch/powerpc/include/asm/mmu-hash64.h | 5 +
> arch/powerpc/include/asm/pgtable-ppc64.h | 31 +----
> arch/powerpc/kernel/io-workarounds.c | 3 +-
> arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 +-
> arch/powerpc/kvm/book3s_hv_rm_mmu.c | 4 +-
> arch/powerpc/mm/Makefile | 1 +
> arch/powerpc/mm/hash_utils_64.c | 16 ++-
> arch/powerpc/mm/hugepage-hash64.c | 185 ++++++++++++++++++++++++++++++
> arch/powerpc/mm/hugetlbpage.c | 31 ++++-
> arch/powerpc/mm/pgtable.c | 38 ++++++
> arch/powerpc/mm/tlb_hash64.c | 5 +-
> arch/powerpc/perf/callchain.c | 2 +-
> arch/powerpc/platforms/pseries/eeh.c | 5 +-
> 13 files changed, 286 insertions(+), 42 deletions(-)
> create mode 100644 arch/powerpc/mm/hugepage-hash64.c
>
> diff --git a/arch/powerpc/include/asm/mmu-hash64.h b/arch/powerpc/include/asm/mmu-hash64.h
> index e187254..a74a3de 100644
> --- a/arch/powerpc/include/asm/mmu-hash64.h
> +++ b/arch/powerpc/include/asm/mmu-hash64.h
> @@ -322,6 +322,11 @@ extern int hash_page(unsigned long ea, unsigned long access, unsigned long trap)
> int __hash_page_huge(unsigned long ea, unsigned long access, unsigned long vsid,
> pte_t *ptep, unsigned long trap, int local, int ssize,
> unsigned int shift, unsigned int mmu_psize);
> +#ifdef CONFIG_TRANSPARENT_HUGEPAGE
> +extern int __hash_page_thp(unsigned long ea, unsigned long access,
> + unsigned long vsid, pmd_t *pmdp, unsigned long trap,
> + int local, int ssize, unsigned int psize);
> +#endif
> extern void hash_failure_debug(unsigned long ea, unsigned long access,
> unsigned long vsid, unsigned long trap,
> int ssize, int psize, int lpsize,
> diff --git a/arch/powerpc/include/asm/pgtable-ppc64.h b/arch/powerpc/include/asm/pgtable-ppc64.h
> index d4e845c..9b81283 100644
> --- a/arch/powerpc/include/asm/pgtable-ppc64.h
> +++ b/arch/powerpc/include/asm/pgtable-ppc64.h
> @@ -345,39 +345,18 @@ static inline void __ptep_set_access_flags(pte_t *ptep, pte_t entry)
> void pgtable_cache_add(unsigned shift, void (*ctor)(void *));
> void pgtable_cache_init(void);
>
> -/*
> - * find_linux_pte returns the address of a linux pte for a given
> - * effective address and directory. If not found, it returns zero.
> - */
> -static inline pte_t *find_linux_pte(pgd_t *pgdir, unsigned long ea)
> -{
> - pgd_t *pg;
> - pud_t *pu;
> - pmd_t *pm;
> - pte_t *pt = NULL;
> -
> - pg = pgdir + pgd_index(ea);
> - if (!pgd_none(*pg)) {
> - pu = pud_offset(pg, ea);
> - if (!pud_none(*pu)) {
> - pm = pmd_offset(pu, ea);
> - if (pmd_present(*pm))
> - pt = pte_offset_kernel(pm, ea);
> - }
> - }
> - return pt;
> -}
> -
> +pte_t *find_linux_pte(pgd_t *pgdir, unsigned long ea, unsigned int *thp);
> #ifdef CONFIG_HUGETLB_PAGE
> pte_t *find_linux_pte_or_hugepte(pgd_t *pgdir, unsigned long ea,
> - unsigned *shift);
> + unsigned *shift, unsigned int *hugepage);
> #else
> static inline pte_t *find_linux_pte_or_hugepte(pgd_t *pgdir, unsigned long ea,
> - unsigned *shift)
> + unsigned *shift,
> + unsigned int *hugepage)
> {
> if (shift)
> *shift = 0;
> - return find_linux_pte(pgdir, ea);
> + return find_linux_pte(pgdir, ea, hugepage);
> }
> #endif /* !CONFIG_HUGETLB_PAGE */
>
> diff --git a/arch/powerpc/kernel/io-workarounds.c b/arch/powerpc/kernel/io-workarounds.c
> index 50e90b7..a9c904f 100644
> --- a/arch/powerpc/kernel/io-workarounds.c
> +++ b/arch/powerpc/kernel/io-workarounds.c
> @@ -70,7 +70,8 @@ struct iowa_bus *iowa_mem_find_bus(const PCI_IO_ADDR addr)
> if (vaddr < PHB_IO_BASE || vaddr >= PHB_IO_END)
> return NULL;
>
> - ptep = find_linux_pte(init_mm.pgd, vaddr);
> + /* we won't find hugepages here */
Explaining why might be a good idea.
> + ptep = find_linux_pte(init_mm.pgd, vaddr, NULL);
> if (ptep == NULL)
> paddr = 0;
> else
> diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> index 8cc18ab..4f2a7dc 100644
> --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
> +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> @@ -683,7 +683,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,
> */
> rcu_read_lock_sched();
> ptep = find_linux_pte_or_hugepte(current->mm->pgd,
> - hva, NULL);
> + hva, NULL, NULL);
> if (ptep && pte_present(*ptep)) {
> pte = kvmppc_read_update_linux_pte(ptep, 1);
> if (pte_write(pte))
> diff --git a/arch/powerpc/kvm/book3s_hv_rm_mmu.c b/arch/powerpc/kvm/book3s_hv_rm_mmu.c
> index 19c93ba..7c8e1ed 100644
> --- a/arch/powerpc/kvm/book3s_hv_rm_mmu.c
> +++ b/arch/powerpc/kvm/book3s_hv_rm_mmu.c
> @@ -27,7 +27,7 @@ static void *real_vmalloc_addr(void *x)
> unsigned long addr = (unsigned long) x;
> pte_t *p;
>
> - p = find_linux_pte(swapper_pg_dir, addr);
> + p = find_linux_pte(swapper_pg_dir, addr, NULL);
And this one.
> if (!p || !pte_present(*p))
> return NULL;
> /* assume we don't have huge pages in vmalloc space... */
> @@ -152,7 +152,7 @@ static pte_t lookup_linux_pte(pgd_t *pgdir, unsigned long hva,
> unsigned long ps = *pte_sizep;
> unsigned int shift;
>
> - ptep = find_linux_pte_or_hugepte(pgdir, hva, &shift);
> + ptep = find_linux_pte_or_hugepte(pgdir, hva, &shift, NULL);
> if (!ptep)
> return __pte(0);
> if (shift)
> diff --git a/arch/powerpc/mm/Makefile b/arch/powerpc/mm/Makefile
> index 3787b61..997deb4 100644
> --- a/arch/powerpc/mm/Makefile
> +++ b/arch/powerpc/mm/Makefile
> @@ -33,6 +33,7 @@ obj-y += hugetlbpage.o
> obj-$(CONFIG_PPC_STD_MMU_64) += hugetlbpage-hash64.o
> obj-$(CONFIG_PPC_BOOK3E_MMU) += hugetlbpage-book3e.o
> endif
> +obj-$(CONFIG_TRANSPARENT_HUGEPAGE) += hugepage-hash64.o
> obj-$(CONFIG_PPC_SUBPAGE_PROT) += subpage-prot.o
> obj-$(CONFIG_NOT_COHERENT_CACHE) += dma-noncoherent.o
> obj-$(CONFIG_HIGHMEM) += highmem.o
> diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c
> index 1f2ebbd..cd3ecd8 100644
> --- a/arch/powerpc/mm/hash_utils_64.c
> +++ b/arch/powerpc/mm/hash_utils_64.c
> @@ -955,7 +955,7 @@ int hash_page(unsigned long ea, unsigned long access, unsigned long trap)
> unsigned long vsid;
> struct mm_struct *mm;
> pte_t *ptep;
> - unsigned hugeshift;
> + unsigned hugeshift, hugepage;
> const struct cpumask *tmp;
> int rc, user_region = 0, local = 0;
> int psize, ssize;
> @@ -1021,7 +1021,7 @@ int hash_page(unsigned long ea, unsigned long access, unsigned long trap)
> #endif /* CONFIG_PPC_64K_PAGES */
>
> /* Get PTE and page size from page tables */
> - ptep = find_linux_pte_or_hugepte(pgdir, ea, &hugeshift);
> + ptep = find_linux_pte_or_hugepte(pgdir, ea, &hugeshift, &hugepage);
> if (ptep == NULL || !pte_present(*ptep)) {
And so's this, since you don't check the hugepage return before
calling pte_present().
> DBG_LOW(" no PTE !\n");
> return 1;
> @@ -1044,6 +1044,12 @@ int hash_page(unsigned long ea, unsigned long access, unsigned long trap)
> ssize, hugeshift, psize);
> #endif /* CONFIG_HUGETLB_PAGE */
>
> +#ifdef CONFIG_TRANSPARENT_HUGEPAGE
> + if (hugepage)
> + return __hash_page_thp(ea, access, vsid, (pmd_t *)ptep,
> + trap, local, ssize, psize);
> +#endif
> +
> #ifndef CONFIG_PPC_64K_PAGES
> DBG_LOW(" i-pte: %016lx\n", pte_val(*ptep));
> #else
> @@ -1149,7 +1155,11 @@ void hash_preload(struct mm_struct *mm, unsigned long ea,
> pgdir = mm->pgd;
> if (pgdir == NULL)
> return;
> - ptep = find_linux_pte(pgdir, ea);
> + /*
> + * We haven't implemented update_mmu_cache_pmd yet. We get called
> + * only for non hugepages. Hence can ignore THP here
Uh.. why? By definition THP will occur in non-hugepage areas.
> + */
> + ptep = find_linux_pte(pgdir, ea, NULL);
> if (!ptep)
> return;
>
> diff --git a/arch/powerpc/mm/hugepage-hash64.c b/arch/powerpc/mm/hugepage-hash64.c
> new file mode 100644
> index 0000000..3f6140d
> --- /dev/null
> +++ b/arch/powerpc/mm/hugepage-hash64.c
> @@ -0,0 +1,185 @@
> +/*
> + * Copyright IBM Corporation, 2013
> + * Author Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of version 2.1 of the GNU Lesser General Public License
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it would be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> + *
> + */
> +
> +/*
> + * PPC64 THP Support for hash based MMUs
> + */
> +#include <linux/mm.h>
> +#include <asm/machdep.h>
> +
> +/*
> + * The linux hugepage PMD now include the pmd entries followed by the address
> + * to the stashed pgtable_t. The stashed pgtable_t contains the hpte bits.
> + * [ secondary group | 3 bit hidx | valid ]. We use one byte per each HPTE entry.
> + * With 16MB hugepage and 64K HPTE we need 256 entries and with 4K HPTE we need
> + * 4096 entries. Both will fit in a 4K pgtable_t.
> + */
> +int __hash_page_thp(unsigned long ea, unsigned long access, unsigned long vsid,
> + pmd_t *pmdp, unsigned long trap, int local, int ssize,
> + unsigned int psize)
> +{
> + unsigned int index, valid;
> + unsigned char *hpte_slot_array;
> + unsigned long rflags, pa, hidx;
> + unsigned long old_pmd, new_pmd;
> + int ret, lpsize = MMU_PAGE_16M;
> + unsigned long vpn, hash, shift, slot;
> +
> + /*
> + * atomically mark the linux large page PMD busy and dirty
> + */
> + do {
> + old_pmd = pmd_val(*pmdp);
> + /* If PMD busy, retry the access */
> + if (unlikely(old_pmd & PMD_HUGE_BUSY))
> + return 0;
> + /* If PMD permissions don't match, take page fault */
> + if (unlikely(access & ~old_pmd))
> + return 1;
> + /*
> + * Try to lock the PTE, add ACCESSED and DIRTY if it was
> + * a write access
> + */
> + new_pmd = old_pmd | PMD_HUGE_BUSY | PMD_HUGE_ACCESSED;
> + if (access & _PAGE_RW)
> + new_pmd |= PMD_HUGE_DIRTY;
> + } while (old_pmd != __cmpxchg_u64((unsigned long *)pmdp,
> + old_pmd, new_pmd));
> + /*
> + * PP bits. PMD_HUGE_USER is already PP bit 0x2, so we only
> + * need to add in 0x1 if it's a read-only user page
> + */
> + rflags = new_pmd & PMD_HUGE_USER;
> + if ((new_pmd & PMD_HUGE_USER) && !((new_pmd & PMD_HUGE_RW) &&
> + (new_pmd & PMD_HUGE_DIRTY)))
> + rflags |= 0x1;
> + /*
> + * PMD_HUGE_EXEC -> HW_NO_EXEC since it's inverted
> + */
> + rflags |= ((new_pmd & PMD_HUGE_EXEC) ? 0 : HPTE_R_N);
> +
> +#if 0 /* FIXME!! */
> + if (!cpu_has_feature(CPU_FTR_COHERENT_ICACHE)) {
> +
> + /*
> + * No CPU has hugepages but lacks no execute, so we
> + * don't need to worry about that case
> + */
> + rflags = hash_page_do_lazy_icache(rflags, __pte(old_pte), trap);
> + }
> +#endif
> + /*
> + * Find the slot index details for this ea, using base page size.
> + */
> + shift = mmu_psize_defs[psize].shift;
> + index = (ea & (HUGE_PAGE_SIZE - 1)) >> shift;
> + BUG_ON(index > 4096);
That needs to be >=, not >. Also you should probably use the existing
#defines to derive this rather than hard coding 4096.
> +
> + vpn = hpt_vpn(ea, vsid, ssize);
> + hash = hpt_hash(vpn, shift, ssize);
> + /*
> + * The hpte hindex are stored in the pgtable whose address is in the
> + * second half of the PMD
> + */
> + hpte_slot_array = *(char **)(pmdp + PTRS_PER_PMD);
Hrm. I gather the contents of the extra pgtable is protected by the
PTE's busy bit. But what synchronization is necessary for the pgtable
pointer - are there any possible races with the hugepage being split?
> + valid = hpte_slot_array[index] & 0x1;
> + if (unlikely(valid)) {
Why is valid unlikely? I think you'd be better off leaving this to
the CPU's dynamic branch prediction.
> + /* update the hpte bits */
> + hidx = hpte_slot_array[index] >> 1;
> + if (hidx & _PTEIDX_SECONDARY)
> + hash = ~hash;
> + slot = (hash & htab_hash_mask) * HPTES_PER_GROUP;
> + slot += hidx & _PTEIDX_GROUP_IX;
> +
> + ret = ppc_md.hpte_updatepp(slot, rflags, vpn,
> + psize, ssize, local);
> + /*
> + * We failed to update, try to insert a new entry.
> + */
> + if (ret == -1) {
> + /*
> + * large pte is marked busy, so we can be sure
> + * nobody is looking at hpte_slot_array. hence we can
> + * safely update this here.
> + */
> + hpte_slot_array[index] = 0;
> + valid = 0;
> + }
> + }
> +
> + if (likely(!valid)) {
> + unsigned long hpte_group;
> +
> + /* insert new entry */
> + pa = pmd_pfn(__pmd(old_pmd)) << PAGE_SHIFT;
> +repeat:
> + hpte_group = ((hash & htab_hash_mask) * HPTES_PER_GROUP) & ~0x7UL;
> +
> + /* clear the busy bits and set the hash pte bits */
> + new_pmd = (new_pmd & ~PMD_HUGE_HPTEFLAGS) | PMD_HUGE_HASHPTE;
> +
> + /*
> + * WIMG bits.
> + * We always have _PAGE_COHERENT enabled for system RAM
> + */
> + rflags |= _PAGE_COHERENT;
> +
> + if (new_pmd & PMD_HUGE_SAO)
> + rflags |= _PAGE_SAO;
> +
> + /* Insert into the hash table, primary slot */
> + slot = ppc_md.hpte_insert(hpte_group, vpn, pa, rflags, 0,
> + psize, lpsize, ssize);
> + /*
> + * Primary is full, try the secondary
> + */
> + if (unlikely(slot == -1)) {
> + hpte_group = ((~hash & htab_hash_mask) *
> + HPTES_PER_GROUP) & ~0x7UL;
> + slot = ppc_md.hpte_insert(hpte_group, vpn, pa,
> + rflags, HPTE_V_SECONDARY,
> + psize, lpsize, ssize);
> + if (slot == -1) {
> + if (mftb() & 0x1)
> + hpte_group = ((hash & htab_hash_mask) *
> + HPTES_PER_GROUP) & ~0x7UL;
> +
> + ppc_md.hpte_remove(hpte_group);
> + goto repeat;
> + }
> + }
> + /*
> + * Hypervisor failure. Restore old pmd and return -1
> + * similar to __hash_page_*
> + */
> + if (unlikely(slot == -2)) {
> + *pmdp = __pmd(old_pmd);
> + hash_failure_debug(ea, access, vsid, trap, ssize,
> + psize, lpsize, old_pmd);
> + return -1;
> + }
> + /*
> + * large pte is marked busy, so we can be sure
> + * nobody is looking at hpte_slot_array. hence we can
> + * safely update this here.
> + */
> + hpte_slot_array[index] = slot << 1 | 0x1;
> + }
> + /*
> + * No need to use ldarx/stdcx here
> + */
> + *pmdp = __pmd(new_pmd & ~PMD_HUGE_BUSY);
> + return 0;
> +}
> diff --git a/arch/powerpc/mm/hugetlbpage.c b/arch/powerpc/mm/hugetlbpage.c
> index 1a6de0a..7f11fa0 100644
> --- a/arch/powerpc/mm/hugetlbpage.c
> +++ b/arch/powerpc/mm/hugetlbpage.c
> @@ -67,7 +67,8 @@ static inline unsigned int mmu_psize_to_shift(unsigned int mmu_psize)
>
> #define hugepd_none(hpd) ((hpd).pd == 0)
>
> -pte_t *find_linux_pte_or_hugepte(pgd_t *pgdir, unsigned long ea, unsigned *shift)
> +pte_t *find_linux_pte_or_hugepte(pgd_t *pgdir, unsigned long ea,
> + unsigned *shift, unsigned int *hugepage)
> {
> pgd_t *pg;
> pud_t *pu;
> @@ -77,6 +78,8 @@ pte_t *find_linux_pte_or_hugepte(pgd_t *pgdir, unsigned long ea, unsigned *shift
>
> if (shift)
> *shift = 0;
> + if (hugepage)
> + *hugepage = 0;
> pg = pgdir + pgd_index(ea);
> if (is_hugepd(pg)) {
> @@ -91,12 +94,24 @@ pte_t *find_linux_pte_or_hugepte(pgd_t *pgdir, unsigned long ea, unsigned *shift
> pm = pmd_offset(pu, ea);
> if (is_hugepd(pm))
> hpdp = (hugepd_t *)pm;
> - else if (!pmd_none(*pm)) {
> + else if (pmd_large(*pm)) {
> + /* THP page */
> + if (hugepage) {
> + *hugepage = 1;
> + /*
> + * This should be ok, except for few
> + * flags. Most of the pte and hugepage
> + * pmd bits overlap. We don't use the
> + * returned value as pte_t in the caller.
> + */
> + return (pte_t *)pm;
> + } else
> + return NULL;
Ah, so this is what prevents callers who don't supply the hugepage
parameter from being horribly broken. Hrm. Seems dangeously subtle
to me.
The parameter name is also really misleading since it's only for
transparent hugepages, not any hugepage.
> + } else if (!pmd_none(*pm)) {
> return pte_offset_kernel(pm, ea);
> }
> }
> }
> -
> if (!hpdp)
> return NULL;
>
> @@ -108,7 +123,8 @@ EXPORT_SYMBOL_GPL(find_linux_pte_or_hugepte);
>
> pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr)
> {
> - return find_linux_pte_or_hugepte(mm->pgd, addr, NULL);
> + /* Only called for HugeTLB pages, hence can ignore THP */
> + return find_linux_pte_or_hugepte(mm->pgd, addr, NULL, NULL);
> }
>
> static int __hugepte_alloc(struct mm_struct *mm, hugepd_t *hpdp,
> @@ -613,8 +629,11 @@ follow_huge_addr(struct mm_struct *mm, unsigned long address, int write)
> struct page *page;
> unsigned shift;
> unsigned long mask;
> -
> - ptep = find_linux_pte_or_hugepte(mm->pgd, address, &shift);
> + /*
> + * Transparent hugepages are handled by generic code. We can skip them
> + * here.
> + */
> + ptep = find_linux_pte_or_hugepte(mm->pgd, address, &shift, NULL);
>
> /* Verify it is a huge page else bail. */
> if (!ptep || !shift)
> diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
> index cf3ca8e..fbff062 100644
> --- a/arch/powerpc/mm/pgtable.c
> +++ b/arch/powerpc/mm/pgtable.c
> @@ -557,3 +557,41 @@ void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr,
> }
>
> #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
> +
> +/*
> + * find_linux_pte returns the address of a linux pte for a given
> + * effective address and directory. If not found, it returns zero.
> + */
> +pte_t *find_linux_pte(pgd_t *pgdir, unsigned long ea, unsigned int *hugepage)
> +{
> + pgd_t *pg;
> + pud_t *pu;
> + pmd_t *pm;
> + pte_t *pt = NULL;
> +
> + if (hugepage)
> + *hugepage = 0;
> + pg = pgdir + pgd_index(ea);
> + if (!pgd_none(*pg)) {
> + pu = pud_offset(pg, ea);
> + if (!pud_none(*pu)) {
> + pm = pmd_offset(pu, ea);
> + if (pmd_large(*pm)) {
> + /* THP page */
> + if (hugepage) {
> + *hugepage = 1;
> + /*
> + * This should be ok, except for few
> + * flags. Most of the pte and hugepage
> + * pmd bits overlap. We don't use the
> + * returned value as pte_t in the caller.
> + */
> + return (pte_t *)pm;
> + } else
> + return NULL;
> + } else if (pmd_present(*pm))
> + pt = pte_offset_kernel(pm, ea);
> + }
> + }
> + return pt;
> +}
> diff --git a/arch/powerpc/mm/tlb_hash64.c b/arch/powerpc/mm/tlb_hash64.c
> index 023ec8a..be0066f 100644
> --- a/arch/powerpc/mm/tlb_hash64.c
> +++ b/arch/powerpc/mm/tlb_hash64.c
> @@ -206,7 +206,10 @@ void __flush_hash_table_range(struct mm_struct *mm, unsigned long start,
> local_irq_save(flags);
> arch_enter_lazy_mmu_mode();
> for (; start < end; start += PAGE_SIZE) {
> - pte_t *ptep = find_linux_pte(mm->pgd, start);
> + /*
> + * We won't find hugepages here.
> + */
> + pte_t *ptep = find_linux_pte(mm->pgd, start, NULL);
> unsigned long pte;
>
> if (ptep == NULL)
> diff --git a/arch/powerpc/perf/callchain.c b/arch/powerpc/perf/callchain.c
> index 74d1e78..578cac7 100644
> --- a/arch/powerpc/perf/callchain.c
> +++ b/arch/powerpc/perf/callchain.c
> @@ -125,7 +125,7 @@ static int read_user_stack_slow(void __user *ptr, void *ret, int nb)
> if (!pgdir)
> return -EFAULT;
>
> - ptep = find_linux_pte_or_hugepte(pgdir, addr, &shift);
> + ptep = find_linux_pte_or_hugepte(pgdir, addr, &shift, NULL);
> if (!shift)
> shift = PAGE_SHIFT;
>
> diff --git a/arch/powerpc/platforms/pseries/eeh.c b/arch/powerpc/platforms/pseries/eeh.c
> index 9a04322..44c931a 100644
> --- a/arch/powerpc/platforms/pseries/eeh.c
> +++ b/arch/powerpc/platforms/pseries/eeh.c
> @@ -261,7 +261,10 @@ static inline unsigned long eeh_token_to_phys(unsigned long token)
> pte_t *ptep;
> unsigned long pa;
>
> - ptep = find_linux_pte(init_mm.pgd, token);
> + /*
> + * We won't find hugepages here
> + */
> + ptep = find_linux_pte(init_mm.pgd, token, NULL);
> if (!ptep)
> return token;
> pa = pte_pfn(*ptep) << PAGE_SHIFT;
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply
* Re: [PATCH -V5 24/25] powerpc: Optimize hugepage invalidate
From: David Gibson @ 2013-04-12 4:21 UTC (permalink / raw)
To: Aneesh Kumar K.V; +Cc: paulus, linuxppc-dev, linux-mm
In-Reply-To: <1365055083-31956-25-git-send-email-aneesh.kumar@linux.vnet.ibm.com>
[-- Attachment #1: Type: text/plain, Size: 11529 bytes --]
On Thu, Apr 04, 2013 at 11:28:02AM +0530, Aneesh Kumar K.V wrote:
> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>
> Hugepage invalidate involves invalidating multiple hpte entries.
> Optimize the operation using H_BULK_REMOVE on lpar platforms.
> On native, reduce the number of tlb flush.
>
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> ---
> arch/powerpc/include/asm/machdep.h | 3 +
> arch/powerpc/mm/hash_native_64.c | 78 ++++++++++++++++++++
> arch/powerpc/mm/pgtable.c | 13 +++-
> arch/powerpc/platforms/pseries/lpar.c | 126 +++++++++++++++++++++++++++++++--
> 4 files changed, 210 insertions(+), 10 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/machdep.h b/arch/powerpc/include/asm/machdep.h
> index 6cee6e0..3bc7816 100644
> --- a/arch/powerpc/include/asm/machdep.h
> +++ b/arch/powerpc/include/asm/machdep.h
> @@ -56,6 +56,9 @@ struct machdep_calls {
> void (*hpte_removebolted)(unsigned long ea,
> int psize, int ssize);
> void (*flush_hash_range)(unsigned long number, int local);
> + void (*hugepage_invalidate)(struct mm_struct *mm,
> + unsigned char *hpte_slot_array,
> + unsigned long addr, int psize);
>
> /* special for kexec, to be called in real mode, linear mapping is
> * destroyed as well */
> diff --git a/arch/powerpc/mm/hash_native_64.c b/arch/powerpc/mm/hash_native_64.c
> index ac84fa6..59f29bf 100644
> --- a/arch/powerpc/mm/hash_native_64.c
> +++ b/arch/powerpc/mm/hash_native_64.c
> @@ -450,6 +450,83 @@ static void native_hpte_invalidate(unsigned long slot, unsigned long vpn,
> local_irq_restore(flags);
> }
>
> +static void native_hugepage_invalidate(struct mm_struct *mm,
> + unsigned char *hpte_slot_array,
> + unsigned long addr, int psize)
> +{
> + int ssize = 0, i;
> + int lock_tlbie;
> + struct hash_pte *hptep;
> + int actual_psize = MMU_PAGE_16M;
> + unsigned int max_hpte_count, valid;
> + unsigned long flags, s_addr = addr;
> + unsigned long hpte_v, want_v, shift;
> + unsigned long hidx, vpn = 0, vsid, hash, slot;
> +
> + shift = mmu_psize_defs[psize].shift;
> + max_hpte_count = HUGE_PAGE_SIZE/(1ul << shift);
> +
> + local_irq_save(flags);
> + for (i = 0; i < max_hpte_count; i++) {
> + /*
> + * 8 bits per each hpte entries
> + * 000| [ secondary group (one bit) | hidx (3 bits) | valid bit]
> + */
> + valid = hpte_slot_array[i] & 0x1;
> + if (!valid)
> + continue;
> + hidx = hpte_slot_array[i] >> 1;
> +
> + /* get the vpn */
> + addr = s_addr + (i * (1ul << shift));
> + if (!is_kernel_addr(addr)) {
> + ssize = user_segment_size(addr);
> + vsid = get_vsid(mm->context.id, addr, ssize);
> + WARN_ON(vsid == 0);
> + } else {
> + vsid = get_kernel_vsid(addr, mmu_kernel_ssize);
> + ssize = mmu_kernel_ssize;
> + }
> +
> + vpn = hpt_vpn(addr, vsid, ssize);
> + hash = hpt_hash(vpn, shift, ssize);
> + if (hidx & _PTEIDX_SECONDARY)
> + hash = ~hash;
> +
> + slot = (hash & htab_hash_mask) * HPTES_PER_GROUP;
> + slot += hidx & _PTEIDX_GROUP_IX;
> +
> + hptep = htab_address + slot;
> + want_v = hpte_encode_avpn(vpn, psize, ssize);
> + native_lock_hpte(hptep);
> + hpte_v = hptep->v;
> +
> + /* Even if we miss, we need to invalidate the TLB */
> + if (!HPTE_V_COMPARE(hpte_v, want_v) || !(hpte_v & HPTE_V_VALID))
> + native_unlock_hpte(hptep);
> + else
> + /* Invalidate the hpte. NOTE: this also unlocks it */
> + hptep->v = 0;
Shouldn't you be clearing the entry from the slot_array once it is
invalidated in the hash table?
> + }
> + /*
> + * Since this is a hugepage, we just need a single tlbie.
> + * use the last vpn.
> + */
> + lock_tlbie = !mmu_has_feature(MMU_FTR_LOCKLESS_TLBIE);
> + if (lock_tlbie)
> + raw_spin_lock(&native_tlbie_lock);
> +
> + asm volatile("ptesync":::"memory");
> + __tlbie(vpn, psize, actual_psize, ssize);
> + asm volatile("eieio; tlbsync; ptesync":::"memory");
> +
> + if (lock_tlbie)
> + raw_spin_unlock(&native_tlbie_lock);
> +
> + local_irq_restore(flags);
> +}
> +
> +
> static void hpte_decode(struct hash_pte *hpte, unsigned long slot,
> int *psize, int *apsize, int *ssize, unsigned long *vpn)
> {
> @@ -678,4 +755,5 @@ void __init hpte_init_native(void)
> ppc_md.hpte_remove = native_hpte_remove;
> ppc_md.hpte_clear_all = native_hpte_clear;
> ppc_md.flush_hash_range = native_flush_hash_range;
> + ppc_md.hugepage_invalidate = native_hugepage_invalidate;
> }
> diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
> index fbff062..386cab8 100644
> --- a/arch/powerpc/mm/pgtable.c
> +++ b/arch/powerpc/mm/pgtable.c
> @@ -433,6 +433,7 @@ void hpte_need_hugepage_flush(struct mm_struct *mm, unsigned long addr,
> {
> int ssize, i;
> unsigned long s_addr;
> + int max_hpte_count;
> unsigned int psize, valid;
> unsigned char *hpte_slot_array;
> unsigned long hidx, vpn, vsid, hash, shift, slot;
> @@ -446,12 +447,18 @@ void hpte_need_hugepage_flush(struct mm_struct *mm, unsigned long addr,
> * second half of the PMD
> */
> hpte_slot_array = *(char **)(pmdp + PTRS_PER_PMD);
> -
> /* get the base page size */
> psize = get_slice_psize(mm, s_addr);
> - shift = mmu_psize_defs[psize].shift;
>
> - for (i = 0; i < HUGE_PAGE_SIZE/(1ul << shift); i++) {
> + if (ppc_md.hugepage_invalidate)
> + return ppc_md.hugepage_invalidate(mm, hpte_slot_array,
> + s_addr, psize);
> + /*
> + * No bluk hpte removal support, invalidate each entry
> + */
> + shift = mmu_psize_defs[psize].shift;
> + max_hpte_count = HUGE_PAGE_SIZE/(1ul << shift);
> + for (i = 0; i < max_hpte_count; i++) {
> /*
> * 8 bits per each hpte entries
> * 000| [ secondary group (one bit) | hidx (3 bits) | valid bit]
> diff --git a/arch/powerpc/platforms/pseries/lpar.c b/arch/powerpc/platforms/pseries/lpar.c
> index 3daced3..5fcc621 100644
> --- a/arch/powerpc/platforms/pseries/lpar.c
> +++ b/arch/powerpc/platforms/pseries/lpar.c
> @@ -45,6 +45,13 @@
> #include "plpar_wrappers.h"
> #include "pseries.h"
>
> +/* Flag bits for H_BULK_REMOVE */
> +#define HBR_REQUEST 0x4000000000000000UL
> +#define HBR_RESPONSE 0x8000000000000000UL
> +#define HBR_END 0xc000000000000000UL
> +#define HBR_AVPN 0x0200000000000000UL
> +#define HBR_ANDCOND 0x0100000000000000UL
> +
>
> /* in hvCall.S */
> EXPORT_SYMBOL(plpar_hcall);
> @@ -339,6 +346,117 @@ static void pSeries_lpar_hpte_invalidate(unsigned long slot, unsigned long vpn,
> BUG_ON(lpar_rc != H_SUCCESS);
> }
>
> +/*
> + * Limit iterations holding pSeries_lpar_tlbie_lock to 3. We also need
> + * to make sure that we avoid bouncing the hypervisor tlbie lock.
> + */
> +#define PPC64_HUGE_HPTE_BATCH 12
> +
> +static void __pSeries_lpar_hugepage_invalidate(unsigned long *slot,
> + unsigned long *vpn, int count,
> + int psize, int ssize)
> +{
> + unsigned long param[9];
> + int i = 0, pix = 0, rc;
> + unsigned long flags = 0;
> + int lock_tlbie = !mmu_has_feature(MMU_FTR_LOCKLESS_TLBIE);
> +
> + if (lock_tlbie)
> + spin_lock_irqsave(&pSeries_lpar_tlbie_lock, flags);
> +
> + for (i = 0; i < count; i++) {
> +
> + if (!firmware_has_feature(FW_FEATURE_BULK_REMOVE)) {
> + pSeries_lpar_hpte_invalidate(slot[i], vpn[i], psize,
> + ssize, 0);
> + } else {
> + param[pix] = HBR_REQUEST | HBR_AVPN | slot[i];
> + param[pix+1] = hpte_encode_avpn(vpn[i], psize, ssize);
> + pix += 2;
> + if (pix == 8) {
> + rc = plpar_hcall9(H_BULK_REMOVE, param,
> + param[0], param[1], param[2],
> + param[3], param[4], param[5],
> + param[6], param[7]);
> + BUG_ON(rc != H_SUCCESS);
> + pix = 0;
> + }
> + }
> + }
> + if (pix) {
> + param[pix] = HBR_END;
> + rc = plpar_hcall9(H_BULK_REMOVE, param, param[0], param[1],
> + param[2], param[3], param[4], param[5],
> + param[6], param[7]);
> + BUG_ON(rc != H_SUCCESS);
> + }
> +
> + if (lock_tlbie)
> + spin_unlock_irqrestore(&pSeries_lpar_tlbie_lock, flags);
> +}
> +
> +static void pSeries_lpar_hugepage_invalidate(struct mm_struct *mm,
> + unsigned char *hpte_slot_array,
> + unsigned long addr, int psize)
> +{
> + int ssize = 0, i, index = 0;
> + unsigned long s_addr = addr;
> + unsigned int max_hpte_count, valid;
> + unsigned long vpn_array[PPC64_HUGE_HPTE_BATCH];
> + unsigned long slot_array[PPC64_HUGE_HPTE_BATCH];
These are really too big to be allocating on the stack. You'd be
better off going direct from the char slot array to the data structure
for H_BULK_REMOVE, rather than introducing this intermediate structure.
> + unsigned long shift, hidx, vpn = 0, vsid, hash, slot;
> +
> + shift = mmu_psize_defs[psize].shift;
> + max_hpte_count = HUGE_PAGE_SIZE/(1ul << shift);
> +
> + for (i = 0; i < max_hpte_count; i++) {
> + /*
> + * 8 bits per each hpte entries
> + * 000| [ secondary group (one bit) | hidx (3 bits) | valid bit]
> + */
> + valid = hpte_slot_array[i] & 0x1;
> + if (!valid)
> + continue;
> + hidx = hpte_slot_array[i] >> 1;
> +
> + /* get the vpn */
> + addr = s_addr + (i * (1ul << shift));
> + if (!is_kernel_addr(addr)) {
> + ssize = user_segment_size(addr);
> + vsid = get_vsid(mm->context.id, addr, ssize);
> + WARN_ON(vsid == 0);
> + } else {
> + vsid = get_kernel_vsid(addr, mmu_kernel_ssize);
> + ssize = mmu_kernel_ssize;
> + }
> +
> + vpn = hpt_vpn(addr, vsid, ssize);
> + hash = hpt_hash(vpn, shift, ssize);
> + if (hidx & _PTEIDX_SECONDARY)
> + hash = ~hash;
> +
> + slot = (hash & htab_hash_mask) * HPTES_PER_GROUP;
> + slot += hidx & _PTEIDX_GROUP_IX;
> +
> + slot_array[index] = slot;
> + vpn_array[index] = vpn;
> + if (index == PPC64_HUGE_HPTE_BATCH - 1) {
> + /*
> + * Now do a bluk invalidate
> + */
> + __pSeries_lpar_hugepage_invalidate(slot_array,
> + vpn_array,
> + PPC64_HUGE_HPTE_BATCH,
> + psize, ssize);
> + index = 0;
> + } else
> + index++;
> + }
> + if (index)
> + __pSeries_lpar_hugepage_invalidate(slot_array, vpn_array,
> + index, psize, ssize);
> +}
> +
> static void pSeries_lpar_hpte_removebolted(unsigned long ea,
> int psize, int ssize)
> {
> @@ -354,13 +472,6 @@ static void pSeries_lpar_hpte_removebolted(unsigned long ea,
> pSeries_lpar_hpte_invalidate(slot, vpn, psize, ssize, 0);
> }
>
> -/* Flag bits for H_BULK_REMOVE */
> -#define HBR_REQUEST 0x4000000000000000UL
> -#define HBR_RESPONSE 0x8000000000000000UL
> -#define HBR_END 0xc000000000000000UL
> -#define HBR_AVPN 0x0200000000000000UL
> -#define HBR_ANDCOND 0x0100000000000000UL
> -
> /*
> * Take a spinlock around flushes to avoid bouncing the hypervisor tlbie
> * lock.
> @@ -446,6 +557,7 @@ void __init hpte_init_lpar(void)
> ppc_md.hpte_removebolted = pSeries_lpar_hpte_removebolted;
> ppc_md.flush_hash_range = pSeries_lpar_flush_hash_range;
> ppc_md.hpte_clear_all = pSeries_lpar_hptab_clear;
> + ppc_md.hugepage_invalidate = pSeries_lpar_hugepage_invalidate;
> }
>
> #ifdef CONFIG_PPC_SMLPAR
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply
* Re: [PATCH -V5 21/25] powerpc: Handle hugepage in perf callchain
From: Aneesh Kumar K.V @ 2013-04-12 5:05 UTC (permalink / raw)
To: David Gibson; +Cc: paulus, linuxppc-dev, linux-mm
In-Reply-To: <20130412013449.GD5065@truffula.fritz.box>
David Gibson <dwg@au1.ibm.com> writes:
> On Thu, Apr 04, 2013 at 11:27:59AM +0530, Aneesh Kumar K.V wrote:
>> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>>
>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
>> ---
>> arch/powerpc/perf/callchain.c | 32 +++++++++++++++++++++-----------
>> 1 file changed, 21 insertions(+), 11 deletions(-)
>>
>> diff --git a/arch/powerpc/perf/callchain.c b/arch/powerpc/perf/callchain.c
>> index 578cac7..99262ce 100644
>> --- a/arch/powerpc/perf/callchain.c
>> +++ b/arch/powerpc/perf/callchain.c
>> @@ -115,7 +115,7 @@ static int read_user_stack_slow(void __user *ptr, void *ret, int nb)
>> {
>> pgd_t *pgdir;
>> pte_t *ptep, pte;
>> - unsigned shift;
>> + unsigned shift, hugepage;
>> unsigned long addr = (unsigned long) ptr;
>> unsigned long offset;
>> unsigned long pfn;
>> @@ -125,20 +125,30 @@ static int read_user_stack_slow(void __user *ptr, void *ret, int nb)
>> if (!pgdir)
>> return -EFAULT;
>>
>> - ptep = find_linux_pte_or_hugepte(pgdir, addr, &shift, NULL);
>> + ptep = find_linux_pte_or_hugepte(pgdir, addr, &shift, &hugepage);
>
> So, this patch pretty much demonstrates that your earlier patch adding
> the optional hugepage argument and making the existing callers pass
> NULL was broken.
>
> Any code which calls this function and doesn't use and handle the
> hugepage return value is horribly broken, so permitting the hugepage
> parameter to be optional is itself broken.
>
> I think instead you need to have an early patch that replaces
> find_linux_pte_or_hugepte with a new, more abstracted interface, so
> that code using it will remain correct when hugepage PMDs become
> possible.
The entire thing could have been simple if we supported only one
hugepage size (this is what sparc ended up doing). I guess we don't want
to do that. Also we want to support 16MB and 16GB, which mean we need
hugepd for 16GB at PGD level. My goal was to keep the hugetlb related
code for both 16MB and 16GB similar and consider THP huge page in a
different bucket.
Let me look at again how best I can simplify find_linux_pte_or_hugepte
-aneehs
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox