libhugetlbfs and sparc64 support

libhugetlbfs and sparc64 support

[Nishanth Aravamudan]

[-- Attachment #1: Type: text/plain, Size: 1980 bytes --]

***********************
Warning: Your file, libhugetlbfs-sparc64.tar.bz2, contains more than 32 files after decompression and cannot be scanned.
***********************


Hello all,

I'm currently trying to add (basic) support for sparc64 to libhugetlbfs.
If nothing else, this will ensure that sparc64 gets adequate testing. I
am finding, though, that using the attached tarball and running `make
func`, which should run through a series of quick functional tests, an
Ultra 60 I have access to hangs hard. To be clear, I'm not ruling out
general instability, but same box is able to build kernels and I'm more
prone to suspect hugepage support in the kernel.

What I'm seeing:

1) Running only the 32-bit tests (by specifying -b32 to
tests/run_test.sh), immediately after readpage finishes, the box hangs
hard, including serial console (has to be power cycled). No oops,
nothing.

2) Running only the 64-bit tests (-b64 to run_tests.sh), immediately
after unlinked_fd finishes, the box hangs similarly. Again, nothing on
the serial console to indicate a problem.

What I'm hoping to get is some help from folks that are sparc-savvy. Is
the box just busted, and maybe I should try to find another? Or are
people able to see similar problems running `make func`.

To reproduce:

Use 2.6.20.2, if possible, to avoid as many known hugepage bugs in
arch-generic code as possible. CONFIG_HUGETLBFS and CONFIG_HUGETLB_PAGE
set (the former selects the latter). Boot with hugepages=20 (or `echo 20
> /proc/sys/vm/nr_hugepages`). Using the attached tarball, run
`make func`. Tell me what happens :)

Note: I fully expect some test to fail, notably icache-hygeiene,
linkhuge and linkshare (the latter because the linker scripts in the
tarball are just dummy copies of the normal .x ones).

If someone could also verify my sparc assembly in elf32_sparc.S and
elf64_sparc.S, I would greatly appreciate it.

Thanks,
Nish

-- 
Nishanth Aravamudan <nacc@us.ibm.com>
IBM Linux Technology Center


[-- Attachment #2: libhugetlbfs-sparc64.tar.bz2 --]
[-- Type: application/octet-stream, Size: 58143 bytes --]

Re: libhugetlbfs and sparc64 support

[David Miller]

From: Nishanth Aravamudan <nacc@us.ibm.com>
Date: Mon, 12 Mar 2007 11:39:20 -0700

> ***********************
> Warning: Your file, libhugetlbfs-sparc64.tar.bz2, contains more than 32 files after decompression and cannot be scanned.
> ***********************

What is it with these extremely stupid email virus scanners, this thing
checks on outgoing emails too?!?!?!

> I'm currently trying to add (basic) support for sparc64 to libhugetlbfs.
> If nothing else, this will ensure that sparc64 gets adequate testing. I
> am finding, though, that using the attached tarball and running `make
> func`, which should run through a series of quick functional tests, an
> Ultra 60 I have access to hangs hard. To be clear, I'm not ruling out
> general instability, but same box is able to build kernels and I'm more
> prone to suspect hugepage support in the kernel.

I'll try to take a look at this, thanks.

Re: libhugetlbfs and sparc64 support

[Nishanth Aravamudan]

On 12.03.2007 [13:40:28 -0700], David Miller wrote:
> From: Nishanth Aravamudan <nacc@us.ibm.com>
> Date: Mon, 12 Mar 2007 11:39:20 -0700
> 
> > ***********************
> > Warning: Your file, libhugetlbfs-sparc64.tar.bz2, contains more than 32 files after decompression and cannot be scanned.
> > ***********************
> 
> What is it with these extremely stupid email virus scanners, this thing
> checks on outgoing emails too?!?!?!

Heh -- well, in case there are any problems, Dave Hansen is letting me
put it up on his space:

http://sr71.net/~dave/linux/nacc/libhugetlbfs-sparc64.tar.bz2

> > I'm currently trying to add (basic) support for sparc64 to
> > libhugetlbfs.  If nothing else, this will ensure that sparc64 gets
> > adequate testing. I am finding, though, that using the attached
> > tarball and running `make func`, which should run through a series
> > of quick functional tests, an Ultra 60 I have access to hangs hard.
> > To be clear, I'm not ruling out general instability, but same box is
> > able to build kernels and I'm more prone to suspect hugepage support
> > in the kernel.
> 
> I'll try to take a look at this, thanks.

Really appreciated.

Thanks,
Nish

-- 
Nishanth Aravamudan <nacc@us.ibm.com>
IBM Linux Technology Center

Re: libhugetlbfs and sparc64 support

[David Miller]

From: Nishanth Aravamudan <nacc@us.ibm.com>
Date: Mon, 12 Mar 2007 14:12:40 -0700

> On 12.03.2007 [13:40:28 -0700], David Miller wrote:
> > I'll try to take a look at this, thanks.
> 
> Really appreciated.

This patch below seems to improve the situation for me
on my SunBlade1500, let me know if you can get farther
on your Ultra60.

This points out a big pet-peeve I have with all of
these hugetlb page table interfaces, every single
platform has the HPAGE_MASK the address parameter,
and missing that is how I keep getting burnt with
hugetlb bugs on sparc64.

I really think the callers should be doing this.

commit bb8236f2b978e8ed84e8b664882905fb5a7dd6c8
Author: David S. Miller <davem@sunset.davemloft.net>
Date:   Mon Mar 12 22:55:39 2007 -0700

    [SPARC64]: Add missing HPAGE_MASK masks on address parameters.
    
    These pte loops all assume the passed in address is HPAGE
    aligned, make sure that is actually true.
    
    Signed-off-by: David S. Miller <davem@davemloft.net>

diff --git a/arch/sparc64/mm/hugetlbpage.c b/arch/sparc64/mm/hugetlbpage.c
index 33fd0b2..00677b5 100644
--- a/arch/sparc64/mm/hugetlbpage.c
+++ b/arch/sparc64/mm/hugetlbpage.c
@@ -248,6 +248,7 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
 	if (!pte_present(*ptep) && pte_present(entry))
 		mm->context.huge_pte_count++;
 
+	addr &= HPAGE_MASK;
 	for (i = 0; i < (1 << HUGETLB_PAGE_ORDER); i++) {
 		set_pte_at(mm, addr, ptep, entry);
 		ptep++;
@@ -266,6 +267,8 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr,
 	if (pte_present(entry))
 		mm->context.huge_pte_count--;
 
+	addr &= HPAGE_MASK;
+
 	for (i = 0; i < (1 << HUGETLB_PAGE_ORDER); i++) {
 		pte_clear(mm, addr, ptep);
 		addr += PAGE_SIZE;

Re: libhugetlbfs and sparc64 support

[Bill Irwin]

On Mon, Mar 12, 2007 at 10:58:04PM -0700, David Miller wrote:
> This patch below seems to improve the situation for me
> on my SunBlade1500, let me know if you can get farther
> on your Ultra60.
> This points out a big pet-peeve I have with all of
> these hugetlb page table interfaces, every single
> platform has the HPAGE_MASK the address parameter,
> and missing that is how I keep getting burnt with
> hugetlb bugs on sparc64.
> I really think the callers should be doing this.

Now you're fixing my hugetlb bugs, too. Thank you so much.

I'll get my sparc64 hardware situation fixed up in a bit.


-- wli