CONFIG_HIGHMEM on PPC

CONFIG_HIGHMEM on PPC

[Mark A. Greer]

Please excuse me if this was talked about recently (or not so
recently)...

Has anyone tried CONFIG_HIGHMEM on PPC lately?

If so, did it work?  Its not working for me (2.4.0-test13-pre3).

When I have 512 MB or less, works fine; as soon as I go over 512 MB, it
[almost] comes up fine, bootp's okay but then complains that there is no
initial console and no init found.  Here are the last few lines from the
console output (run with 1GB of memory with the memory ctlr having the
memory set up correctly and with the memory all contiguous):

Linux Tulip driver version 0.9.11 (November 3, 2000)
eth0: Digital DS21143 Tulip rev 65 at 0x7fff00, 00:01:AF:02:9D:99, IRQ
18.
eth0:  EEPROM default media type Autosense.
eth0:  Index #0 - Media MII (#11) described by a 21142 MII PHY (3)
block.
eth0:  MII transceiver #8 config 1000 status 7809 advertising 01e1.
eth1: Digital DS21143 Tulip rev 65 at 0x7ffe80, 00:01:AF:02:9D:98, IRQ
29.
eth1:  EEPROM default media type Autosense.
eth1:  Index #0 - Media MII (#11) described by a 21142 MII PHY (3)
block.
eth1:  MII transceiver #8 config 1000 status 7809 advertising 01e1.
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP, IGMP
IP: routing cache hash table of 8192 buckets, 64Kbytes
TCP: Hash tables configured (established 65536 bind 65536)
Sending BOOTP requests..... OK
IP-Config: Got BOOTP answer from 192.168.1.1, my address is 192.168.1.3
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
devfs: v0.102 (20000622) Richard Gooch (rgooch@atnf.csiro.au)
devfs: boot_options: 0x2
Looking up port of RPC 100003/2 on 192.168.1.1
Looking up port of RPC 100005/2 on 192.168.1.1
VFS: Mounted root (nfs filesystem) readonly.
Freeing unused kernel memory: 184k init 4k prep 4k openfirmware
Warning: unable to open an initial console.
Kernel panic: No init found.  Try passing init= option to kernel.
Rebooting in 180 seconds..


I need to look closer at ioremap_base, etc. but I have it set to
0xf0000000 which where I have BAT 0 set to map 1-1 up to 0xffffffff
(various devices are from 0xf000000 on up; PCI memory space at
0x80000000; PCI I/O space at 0xfc000000).

I'd appreciate any thoughts/suggestions.

Thanks,

Mark


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[Dan Malek]

"Mark A. Greer" wrote:

> Has anyone tried CONFIG_HIGHMEM on PPC lately?

I think it is broken and I have also stumbled into a fix for it.

When HIGHMEM is used, memory pages are mapped to virtual addresses
that are not 1:1 mapped.  Our simple virt_to_phys macros won't work
with HIGHMEM pages.  The 4xx and 8xx processor (and in my test
kernel all processors) map the virt_to_phys to the 'iopa' function
in arch/ppc/mm/init.c.  This way the actual page table entry is
tracked down for a virtual address.

Change your virt_to_phys, virt_to_bus macros in include/asm-ppc/io.h
to call 'iopa' (and do the PCI_DRAM_OFFSET adjustment as necessary)
and let me know what happens.  I'm working on the reverse, using
the mm_ptov() but I don't have it done yet.  If you are using the
reverse (bus_to_virt, etc.) functions, let me know and I'll try to
find a quick hack.

I would like to know who works on APUS as well.  In making these
VM changes, I certainly broke the APUS code (because I stole from
there and made it all common :-).  I would like to work with someone
to straighten this out as I have no way to test it.

Thanks.

	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

HTTP daemon required.

[Srinivas Rao.M]

Hi,
  I want to port an HTTP daemon (preferably version 1.0) on to my target
which is running a FreeBsd 4.4 compatible network stack. The reason why i
want to port is that i want to make some of the configuration changes of
the device through the WebBrowser. Can any one tell me where can i get
the source code of HTTP server.

give me some pointers and URL so that i can dig out some more information
on this.

-early reply is appreciated.
regards
Srini...

--

A platitude is simply a truth repeated till people get tired of hearing it.
-Stanley Baldwin


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[Roman Zippel]

Hi,

On Thu, 25 Jan 2001, Dan Malek wrote:

> When HIGHMEM is used, memory pages are mapped to virtual addresses
> that are not 1:1 mapped.  Our simple virt_to_phys macros won't work
> with HIGHMEM pages.

And they don't have to. If you have a highmem page, use kmap to get
temporary virtual mapping. On the other hand most of the drivers should
never see a highmem page.

> I would like to know who works on APUS as well.

I do currently.

bye, Roman


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[Dan Malek]

Roman Zippel wrote:

> And they don't have to.

So, why do we have so damn many ways and hacks to do stuff like
this?  If a page or I/O is mapped into the VM space, there should
be a set of common functions to manage this space, and exactly one
function to do the virt_to_phys.

> .... If you have a highmem page, use kmap to get
> temporary virtual mapping. On the other hand most of the drivers should
> never see a highmem page.

On the other hand, if highmem was designed properly the kernel
should just fault in the needed pages and virt_to_phys and friends
should just work in this space, too.  Yet another set of functions
to use and restrictions just because a physical page exists beyond
some arbitrary boundary?  Geeze, I thought we learned from Mush-DOS
back in the mid-80's this was not a good design........


> I do currently.

I removed the #ifdef APUS in the virt_to_phys and friends macros
and made everyone call iopa which is going to move to a new file
called arch/ppc/mm/ioremap.c (like other architectures do).  I
don't think the mm_ptov() function works on anything because the
kmap_chunks is gone (at least in the linuxppc_2_5 I am using).
Perhaps I'm just missing some updates from you.  I'll delete
these functions from apus_setup.c since they are common to everyone
now.  I'm almost done with this first phase of IBM4xx merge, which
I will probably start checking in later today.  I'm just testing
on as many other platforms as I can to ensure I didn't break other
stuff.

Thanks.


	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[Roman Zippel]

Hi,

On Thu, 25 Jan 2001, Dan Malek wrote:

> > .... If you have a highmem page, use kmap to get
> > temporary virtual mapping. On the other hand most of the drivers should
> > never see a highmem page.
>
> On the other hand, if highmem was designed properly the kernel
> should just fault in the needed pages and virt_to_phys and friends
> should just work in this space, too.  Yet another set of functions
> to use and restrictions just because a physical page exists beyond
> some arbitrary boundary?  Geeze, I thought we learned from Mush-DOS
> back in the mid-80's this was not a good design........

The problem is the ia32 guys want to address 64GByte, what requires a
64bit physical address, what nobody can handle on 32bit Systems.

> I removed the #ifdef APUS in the virt_to_phys and friends macros
> and made everyone call iopa which is going to move to a new file
> called arch/ppc/mm/ioremap.c (like other architectures do).  I
> don't think the mm_ptov() function works on anything because the
> kmap_chunks is gone (at least in the linuxppc_2_5 I am using).
> Perhaps I'm just missing some updates from you.  I'll delete
> these functions from apus_setup.c since they are common to everyone
> now.  I'm almost done with this first phase of IBM4xx merge, which
> I will probably start checking in later today.  I'm just testing
> on as many other platforms as I can to ensure I didn't break other
> stuff.

Please be careful with what you're doing, otherwise you have to rewrite it
again for the real 2.5.
For what exactly do you need such generic virt_to_phys? AFAIK 2.5 will go
more page oriented, so drivers get a set of pages to work on. That will
mean bounce buffer management will be moved into the drivers. So I expect
at some point virt_to_phys will become obsolete or at least private to the
architecture specific part. Either everything is done with pages or one
has to store a virtual/physical address pair. Doing a generic conversion
functions for the latter is often slower, so it's currently expected that
drivers just store it themselves.
That's at least my understanding of the developments to come, what can be
of course completely wrong. :-) To be sure you had to ask one the kernel
gurus.

bye, Roman


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[Gabriel Paubert]

On Thu, 25 Jan 2001, Dan Malek wrote:

>
> Roman Zippel wrote:
>
> > And they don't have to.
>
> So, why do we have so damn many ways and hacks to do stuff like
> this?  If a page or I/O is mapped into the VM space, there should
> be a set of common functions to manage this space, and exactly one
> function to do the virt_to_phys.
>
> > .... If you have a highmem page, use kmap to get
> > temporary virtual mapping. On the other hand most of the drivers should
> > never see a highmem page.
>
> On the other hand, if highmem was designed properly the kernel
> should just fault in the needed pages and virt_to_phys and friends
> should just work in this space, too.  Yet another set of functions
> to use and restrictions just because a physical page exists beyond
> some arbitrary boundary?  Geeze, I thought we learned from Mush-DOS
> back in the mid-80's this was not a good design........

It's quite hard when there are more bits in a physical address bits than
in a pointer. The 7450 have 36 address lines (like PPro+ BTW). Has anybody
any docs on how the MMU has been modified to handle the extra bits on the
7450. Motorola has mostly marketing pamphlets on its web site.

The only port which is simple is the 68k AFAICT: use moves instructions
for {get,put}_user and map all physical memory in the kernel even if you
have 3 gigs. Using such a simple trick for processors which do not allow
to (easily or not) access alternate address spaces is not an option. Ah,
if only there were 500MHz 68060 ;-)

Of course, 64 bit architectures will solve the problem ... until bloatware
makes 2^63 bytes too small even for the simplest applications and the
brand new 128 bit architectures start to appear. Now, assuming that you
can store one bit in one atom of hydrogen, the weight of 2^128 bytes of
memory would be in the range of a few 10^9 tons, hardly practical for
mobile systems ;-)

	Regards,
	Gabriel.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[Dan Malek]

Roman Zippel wrote:

> The problem is the ia32 guys want to address 64GByte, what requires a
> 64bit physical address, what nobody can handle on 32bit Systems.

Heh....64Gbyte is nothing, just a few more bits beyond 32....you
don't need 64-bits for this.  In fact, I had a revelation last
night where I thought we could even do this quite transparently
on PowerPCs with VSIDs or even context registers, anything that
implictly gives us a few more bits of virtual address.  Later
on that.....

Of course, the quick solution today would be to just move the
kernel virtual address to a lower address.........no need for
highmem config at all.  It wouldn't give you the 64Gbyte, but it
would solve the immediate problem of 1G real memory.

> Please be careful with what you're doing, otherwise you have to rewrite it
> again for the real 2.5.

All I am doing is cleaning up a bunch of the PowerPC MM initialization
and mapping functions.  Just getting rid of ifdefs, duplication of
code and stuff like that.  I suspect we will be rewriting in 2.5,
and there is no sense to do it multiple times for different PPC
processor variants.

> For what exactly do you need such generic virt_to_phys? AFAIK 2.5 will go
> more page oriented, so drivers get a set of pages to work on.

All of the embedded PowerPC, and the new high end PowerPC all work
with page mapped TLBs.  The 'middle' processors, 6xx/7xx/74xx have
the BAT register enhancement that appears to be going away.  On the
embedded processors, all I/O is mapped through page tables, which
usually don't have 1:1 mapping.  The existing virt_to_phys and such
with just the arithmetic adjustment don't work on those.  Some
systems have big holes in the physical space, including physical
addresses that map "under" the fixed kernel virtual addresses.  You
have to get to those via page tables as well.

I don't know what you mean by more "page oriented" as that is the
way it works now.  We just use BATs in some cases for efficiency, but
you don't have to.

> ..... That will
> mean bounce buffer management will be moved into the drivers.

Oh geeze.......This kind of stuff should be very generic and even
be hidden.  If drivers want to know for some performance reason
that's OK, but they shouldn't have to know.

> ....... Either everything is done with pages or one
> has to store a virtual/physical address pair. Doing a generic conversion
> functions for the latter is often slower, so it's currently expected that
> drivers just store it themselves.

Oh great, yet another technology path we think we are inventing again.
I hope whoever gets blessed by Linus to work on this takes a serious
look at over 20 years of research and implementation by others.  Oh
well, I'll just hack my little mapping functions :-).


	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[David Edelsohn]

>>>>> Dan Malek writes:

Dan> Heh....64Gbyte is nothing, just a few more bits beyond 32....you
Dan> don't need 64-bits for this.  In fact, I had a revelation last
Dan> night where I thought we could even do this quite transparently
Dan> on PowerPCs with VSIDs or even context registers, anything that
Dan> implictly gives us a few more bits of virtual address.  Later
Dan> on that.....

	Yes, that is the exact purpose of the PowerPC addressing model.
While one user context is limited to 32-bit addresses, the system can map
a larger physical address range into the process's virtual address space.
The process even could handle this with OS cooperation using overlays.

David

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[Dan Malek]

Gabriel Paubert wrote:

> ..... Has anybody
> any docs on how the MMU has been modified to handle the extra bits on the
> 7450.

Yes, I have one and I am working on it.  Although I have been spending
time at the lower end of the processor scale, my changes are directly
applicable to the 7450 enhancements as well.  Of course, we then
have the "common kernel" problem :-).  If I build something to
take advantage of the 7450 enhancements, it may not boot on other
systems.

> The only port which is simple is the 68k

Lots of people complain about it, but I'll agree the 68k MMU
design and supporting instructions is still the best.

> ...... Using such a simple trick for processors which do not allow
> to (easily or not) access alternate address spaces is not an option.

You can easily make the PowerPC have this same alternate address
design.  Either people forget or have never known the value of these
large virtual alternate address space designs, and simply jump to
the common space design we have in Linux.

> ............ Now, assuming that you
> can store one bit in one atom of hydrogen, the weight of 2^128 bytes of
> memory would be in the range of a few 10^9 tons, hardly practical for
> mobile systems ;-)

Of course, you know that 1 bits weight more than 0 bits, right?  I
proved it once.  I wrote a streamer tape with all 0s, weighed it,
then wrote it with all 1s.  It weighed slightly more with all 1s :-).


	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[Roman Zippel]

Hi,

On Thu, 25 Jan 2001, Dan Malek wrote:

> On the
> embedded processors, all I/O is mapped through page tables, which
> usually don't have 1:1 mapping.  The existing virt_to_phys and such
> with just the arithmetic adjustment don't work on those.

And it doesn't really have to. virt_to_bus() might be a better place for
this. virt_to_phys() is at least on ia32 a quite speed sensitive function,
that only needs to work on normal low memory.

> Some
> systems have big holes in the physical space, including physical
> addresses that map "under" the fixed kernel virtual addresses.

With holes do you mean several memory chunks? These should be handled with
memory zones.

> I don't know what you mean by more "page oriented" as that is the
> way it works now.  We just use BATs in some cases for efficiency, but
> you don't have to.

I mean drivers get a pointer to a "struct page", what they can use to
get a virtual or physical address.

bye, Roman


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[Gabriel Paubert]

On Thu, 25 Jan 2001, Dan Malek wrote:

> Gabriel Paubert wrote:
>
> > ..... Has anybody
> > any docs on how the MMU has been modified to handle the extra bits on the
> > 7450.
>
> Yes, I have one and I am working on it.  Although I have been spending
> time at the lower end of the processor scale, my changes are directly
> applicable to the 7450 enhancements as well.  Of course, we then
> have the "common kernel" problem :-).  If I build something to
> take advantage of the 7450 enhancements, it may not boot on other
> systems.

If you build an x86 kernel for 36 bit extension, it won't either boot on
earlier processors, and page tables are twice the size for 1/8 increase
in the number of address bits.

>
> > The only port which is simple is the 68k
>
> Lots of people complain about it, but I'll agree the 68k MMU
> design and supporting instructions is still the best.

It was rather clean, except for the fact that it changed too much between
processors: 68020/68030/68040 are all different. I think
MMU[68060]==MMU[68040] but that's about all (never had a 68060).

>
> > ...... Using such a simple trick for processors which do not allow
> > to (easily or not) access alternate address spaces is not an option.
>
> You can easily make the PowerPC have this same alternate address
> design.  Either people forget or have never known the value of these
> large virtual alternate address space designs, and simply jump to
> the common space design we have in Linux.

Yes, I am aware of this and have been thinking on this for about one year.
Just never had time to write it and my plans are actually more ambitious.
I'm getting two Macs to play on (one from the institute and a notebook for
me) from next week so I can test PMAC/Prep/CHRP. I wanted to first work on
a common zImage for all these machines however, because it was necessary
for a few other things that I have in mind. But I'm only doing this in my
copious (cough) spare time (with a wife, a 3 year old son and a second one
coming soon).

The MM design of which Linus is so proud is actually not well suited to
6xx/7xx processors. Can't speak about high end IBM processors.

	Regards,
	Gabriel.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[David Edelsohn]

>>>>> Gabriel Paubert writes:

Gabriel> The MM design of which Linus is so proud is actually not well suited to
Gabriel> 6xx/7xx processors. Can't speak about high end IBM processors.

	For better or worse, Linux's hardware abstraction layer is the
IA-32 architecture.  And no, that does not map well to other architectures
without an 3-level (?), forward page table.

David

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[Gabriel Paubert]

On Thu, 25 Jan 2001, Dan Malek wrote:

>
> Roman Zippel wrote:
>
> > The problem is the ia32 guys want to address 64GByte, what requires a
> > 64bit physical address, what nobody can handle on 32bit Systems.
>
> Heh....64Gbyte is nothing, just a few more bits beyond 32....you
> don't need 64-bits for this.  In fact, I had a revelation last
> night where I thought we could even do this quite transparently
> on PowerPCs with VSIDs or even context registers, anything that
> implictly gives us a few more bits of virtual address.  Later
> on that.....

You've seen the light, yes it is the way to go at least on 6xx/7xx[x]...

> Of course, the quick solution today would be to just move the
> kernel virtual address to a lower address.........no need for
> highmem config at all.  It wouldn't give you the 64Gbyte, but it
> would solve the immediate problem of 1G real memory.

Or even up to 3Gb easily, while at the same time increasing the virtual
address space of each process. For 36 bit extension, you absolutely need
kmap style kludges, however. 32 bit is nothing, 4Gb of RAM is cheap these
days. OTOH if Motorola has come with real 64 bit processors instead of
this 36 bit band-aid, the point would be moot.

Note that some data has to be accessible at all times from the kernel, and
having less than 1Gb directly accessible while you have 64Gb of RAM
(that's just about 1.5%) is not a healthy situation on x86.

> All I am doing is cleaning up a bunch of the PowerPC MM initialization
> and mapping functions.  Just getting rid of ifdefs, duplication of
> code and stuff like that.  I suspect we will be rewriting in 2.5,
> and there is no sense to do it multiple times for different PPC
> processor variants.
>
> > For what exactly do you need such generic virt_to_phys? AFAIK 2.5 will go
> > more page oriented, so drivers get a set of pages to work on.
>
> All of the embedded PowerPC, and the new high end PowerPC all work
> with page mapped TLBs.  The 'middle' processors, 6xx/7xx/74xx have
> the BAT register enhancement that appears to be going away.  On the

IBM and Motorola are diverging on this point: the 755 has 8 BATS, but
Power4 doesn't have them. I'm still looking for docs on the evolution of
the MMU, this could be important for implementation decisions. But
Motorola could well kill BATS on the next chip depending on the phase of
moon or some horoscope or anything else...

> I don't know what you mean by more "page oriented" as that is the
> way it works now.  We just use BATs in some cases for efficiency, but
> you don't have to.

BATS are a good idea, but variable page size are another, superior IMHO,
possibility. It seems to be what Power4 has from what David said a few
days ago. I don't know how they mix variable page size with hash tables
and failed to find any detailed documentation...

> > ....... Either everything is done with pages or one
> > has to store a virtual/physical address pair. Doing a generic conversion
> > functions for the latter is often slower, so it's currently expected that
> > drivers just store it themselves.
>
> Oh great, yet another technology path we think we are inventing again.
> I hope whoever gets blessed by Linus to work on this takes a serious
> look at over 20 years of research and implementation by others.  Oh
> well, I'll just hack my little mapping functions :-).

Indeed.

	Regards,
	Gabriel.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[Roman Zippel]

Hi,

On Thu, 25 Jan 2001, Gabriel Paubert wrote:

> > Heh....64Gbyte is nothing, just a few more bits beyond 32....you
> > don't need 64-bits for this.  In fact, I had a revelation last
> > night where I thought we could even do this quite transparently
> > on PowerPCs with VSIDs or even context registers, anything that
> > implictly gives us a few more bits of virtual address.  Later
> > on that.....
>
> You've seen the light, yes it is the way to go at least on 6xx/7xx[x]...

Could someone enlighten me, too? :)

bye, Roman


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[David Edelsohn]

>>>>> Gabriel Paubert writes:

Gabriel> BATS are a good idea, but variable page size are another, superior IMHO,
Gabriel> possibility. It seems to be what Power4 has from what David said a few
Gabriel> days ago. I don't know how they mix variable page size with hash tables
Gabriel> and failed to find any detailed documentation...

	The variable page sizes are determined on a segment by segment
basis.  The segment type determines how to interpret the page table --
whether it maps 4KB or 16MB.

David

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: CONFIG_HIGHMEM on PPC

[Frank Rowand]

David Edelsohn wrote:
>
> >>>>> Gabriel Paubert writes:
>
> Gabriel> BATS are a good idea, but variable page size are another, superior IMHO,
> Gabriel> possibility. It seems to be what Power4 has from what David said a few
> Gabriel> days ago. I don't know how they mix variable page size with hash tables
> Gabriel> and failed to find any detailed documentation...
>
>         The variable page sizes are determined on a segment by segment
> basis.  The segment type determines how to interpret the page table --
> whether it maps 4KB or 16MB.

The 405 processors have page sizes of
  1k
  4k
 16k
 64k
256k
  1m
  4m
 16m

The 440 processors are the same except
  4m removed
256m added

The page size is determined by a field in the TLB entry.

-Frank
--
Frank Rowand <frank_rowand@mvista.com>
MontaVista Software, Inc

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/