[Linux-ia64] 2.5.45 with CONFIG_NUMA=y

[Linux-ia64] 2.5.45 with CONFIG_NUMA=y

[Xavier Bru]

Hi,

Booting the 2.5.45 kernel on our numa platform, we hit 2 problems:

. Current settings for DIG are : 512MB/bank, 16GB/node,(mmzone.h)  but our
platform can support 64 GB/node (Even with less memory, bios maps
each node memory modulo 64GB). 
As it has been cleanly coded :-), we just changed DIG_BANKSHIFT to 31 
(2GB banks) and system runs. 
Should these values bee incremented in the future ?

. We can't get X11 working (Xserver loops in user mode). Note that 
it works OK with 2.5.45 without DISCONTIG_MEM set. Having a look to the
mappings in /proc/ and the XFree86 log, we saw no difference. Any idea 
?

thanks in advance for your help.

Xavier

-- 

 Sincères salutations.
_____________________________________________________________________
 
Xavier BRU                 BULL ISD/R&D/INTEL office:     FREC B1-422
tel : +33 (0)4 76 29 77 45                    http://www-frec.bull.fr
fax : +33 (0)4 76 29 77 70                 mailto:Xavier.Bru@bull.net
addr: BULL, 1 rue de Provence, BP 208, 38432 Echirolles Cedex, FRANCE
_____________________________________________________________________

Re: [Linux-ia64] 2.5.45 with CONFIG_NUMA=y

[Jesse Barnes]

On Tue, Nov 26, 2002 at 06:54:36PM +0100, Xavier Bru wrote:
> 
> Hi,
> 
> Booting the 2.5.45 kernel on our numa platform, we hit 2 problems:
> 
> . Current settings for DIG are : 512MB/bank, 16GB/node,(mmzone.h)  but our
> platform can support 64 GB/node (Even with less memory, bios maps
> each node memory modulo 64GB). 
> As it has been cleanly coded :-), we just changed DIG_BANKSHIFT to 31 
> (2GB banks) and system runs. 
> Should these values bee incremented in the future ?

Maybe that should be an option for the kernel configuration system?

> . We can't get X11 working (Xserver loops in user mode). Note that 
> it works OK with 2.5.45 without DISCONTIG_MEM set. Having a look to the
> mappings in /proc/ and the XFree86 log, we saw no difference. Any idea 
> ?

Does X initialize your card correctly?  I know that scanning /dev/mem
for video cards (which is what XFree86 does, IIRC) can be problematic on
some platforms...

Jesse

Re: [Linux-ia64] 2.5.45 with CONFIG_NUMA=y

[Xavier Bru]

Hi Jesse
Thanks a lot for your answers.

Jesse Barnes writes:
 > On Tue, Nov 26, 2002 at 06:54:36PM +0100, Xavier Bru wrote:
 > > 
 > > Hi,
 > > 
 > > Booting the 2.5.45 kernel on our numa platform, we hit 2 problems:
 > > 
 > > . Current settings for DIG are : 512MB/bank, 16GB/node,(mmzone.h)  but our
 > > platform can support 64 GB/node (Even with less memory, bios maps
 > > each node memory modulo 64GB). 
 > > As it has been cleanly coded :-), we just changed DIG_BANKSHIFT to 31 
 > > (2GB banks) and system runs. 
 > > Should these values bee incremented in the future ?
 > 
 > Maybe that should be an option for the kernel configuration system?

This should be OK for us.

 > 
 > > . We can't get X11 working (Xserver loops in user mode). Note that 
 > > it works OK with 2.5.45 without DISCONTIG_MEM set. Having a look to the
 > > mappings in /proc/ and the XFree86 log, we saw no difference. Any idea 
 > > ?
 > 
 > Does X initialize your card correctly?  I know that scanning /dev/mem
 > for video cards (which is what XFree86 does, IIRC) can be problematic on
 > some platforms...

Seems that we use the VGA mode through /dev/mem (No driver is actually 
loaded). 
Are there some known problems with /dev/mem and DISCONTIG_MEM ? 
Do you think we should have more chances validating Frame buffer support and the
card driver ? (I think platform has no AGP support).
 > 
 > Jesse
Thanks again.
Xavier

-- 

 Sincères salutations.
_____________________________________________________________________
 
Xavier BRU                 BULL ISD/R&D/INTEL office:     FREC B1-422
tel : +33 (0)4 76 29 77 45                    http://www-frec.bull.fr
fax : +33 (0)4 76 29 77 70                 mailto:Xavier.Bru@bull.net
addr: BULL, 1 rue de Provence, BP 208, 38432 Echirolles Cedex, FRANCE
_____________________________________________________________________

Re: [Linux-ia64] 2.5.45 with CONFIG_NUMA=y

[Jesse Barnes]

On Wed, Nov 27, 2002 at 05:22:04PM +0100, Xavier Bru wrote:
>  > > . Current settings for DIG are : 512MB/bank, 16GB/node,(mmzone.h)  but our
>  > > platform can support 64 GB/node (Even with less memory, bios maps
>  > > each node memory modulo 64GB). 
>  > > As it has been cleanly coded :-), we just changed DIG_BANKSHIFT to 31 
>  > > (2GB banks) and system runs. 
>  > > Should these values bee incremented in the future ?
>  > 
>  > Maybe that should be an option for the kernel configuration system?
> 
> This should be OK for us.

Wanna come up with a patch? :)  It could just be something like 'MB per
bank' or somesuch, only visible when DIG and DISCONTIG are turned on.

> Seems that we use the VGA mode through /dev/mem (No driver is actually 
> loaded). 
> Are there some known problems with /dev/mem and DISCONTIG_MEM ? 

I don't think there are discontig mem issues really, but platform issues
possibly.  On our platform for example, X might go looking in /dev/mem
for a list of PCI cards and then use the info in the PCI config space
incorrectly (e.g. IRQ values).  Other platforms probably have this issue
too, not sure about yours though.

> Do you think we should have more chances validating Frame buffer support and the
> card driver ? (I think platform has no AGP support).

Possibly.  I know that some of the frame buffer drivers were written to
expect the card to be uninitialized so that they'll work on non-x86
platforms.

Jesse