GFP_DMA32 and PAE x86 machines

GFP_DMA32 and PAE x86 machines

[Dave Airlie]

Just a question,

So we have GFP_DMA32 on x86-64 to gives us memory below 4G? so we not
have the same on PAE based x86 machines so I can use a consistent API
from the drm?

Otherwise it'll be stick some ifdefs in the drm to deal with it..

Dave.

Re: GFP_DMA32 and PAE x86 machines

[H. Peter Anvin]

Dave Airlie wrote:
> Just a question,
> 
> So we have GFP_DMA32 on x86-64 to gives us memory below 4G? so we not
> have the same on PAE based x86 machines so I can use a consistent API
> from the drm?
> 
> Otherwise it'll be stick some ifdefs in the drm to deal with it..
> 

Well, all lowmem is below 4 GB on i386...

	-hpa

Re: GFP_DMA32 and PAE x86 machines

[Dave Airlie]

Depending on split all lowmem is below 1GB which isn't exactly
optimal, I'd llike all 4GB for DMA.

Dave.

On 5/31/07, H. Peter Anvin <hpa@zytor.com> wrote:
> Dave Airlie wrote:
> > Just a question,
> >
> > So we have GFP_DMA32 on x86-64 to gives us memory below 4G? so we not
> > have the same on PAE based x86 machines so I can use a consistent API
> > from the drm?
> >
> > Otherwise it'll be stick some ifdefs in the drm to deal with it..
> >
>
> Well, all lowmem is below 4 GB on i386...
>
> 	-hpa
>

Re: GFP_DMA32 and PAE x86 machines

[H. Peter Anvin]

Dave Airlie wrote:
> Depending on split all lowmem is below 1GB which isn't exactly
> optimal, I'd llike all 4GB for DMA.

Yes, but creating new memory zones is expensive.

	-hpa

Re: GFP_DMA32 and PAE x86 machines

[Andi Kleen]

On Thursday 31 May 2007 00:07:45 Dave Airlie wrote:
> Depending on split all lowmem is below 1GB which isn't exactly
> optimal, I'd llike all 4GB for DMA.

Well it would be for a quite specialized limited use case: 
- Memory the kernel doesn't need to map (after all kmap is evil) 
- You need more than 500MB or so.
- 32bit kernel and user cannot run 64bit kernel
- Machine has >3GB of RAM 

Is there clear evidence that is a common issue? If it is just a "would
be nice to have in theory" the cost of doing a GFP_DMA32 for i386 would be
probably not worth doing it. If it's a common issue it might be 
considered, although the "here's a nickle. buy yourself a 64bit CPU"
strategy would also sound attractive. 

Please give a very very strong rationale why you want it. Did you actually
run into such a situation yourself yet?

-Andi

Re: GFP_DMA32 and PAE x86 machines

[H. Peter Anvin]

Andi Kleen wrote:
> On Thursday 31 May 2007 00:07:45 Dave Airlie wrote:
>> Depending on split all lowmem is below 1GB which isn't exactly
>> optimal, I'd llike all 4GB for DMA.
> 
> Well it would be for a quite specialized limited use case: 
> - Memory the kernel doesn't need to map (after all kmap is evil) 
> - You need more than 500MB or so.
> - 32bit kernel and user cannot run 64bit kernel
> - Machine has >3GB of RAM 

There aren't really a whole lot of 32-bit boxen with > 4 GB of RAM (for
4 GB or less -- 2-3 GB or less if the chipset supports remapping --
normal HIGHMEM is all in the required area), and the few that are tend
to be running specialized applications (git.kernel.org is one of those
machines.)

Since it appears that legacy memory standards, SDRAM and DDR-1, aren't
dropping in price like DDR-2 is, I suspect that will more or less remain
that way.

The other problem -- which is a problem in general -- is the
proliferation of DMA masks, and that doesn't look like it's going to get
any better.

	-hpa

Re: GFP_DMA32 and PAE x86 machines

[Dave Airlie]

On 5/31/07, Andi Kleen <ak@suse.de> wrote:
> On Thursday 31 May 2007 00:07:45 Dave Airlie wrote:
> > Depending on split all lowmem is below 1GB which isn't exactly
> > optimal, I'd llike all 4GB for DMA.
>
> Well it would be for a quite specialized limited use case:
> - Memory the kernel doesn't need to map (after all kmap is evil)
> - You need more than 500MB or so.
> - 32bit kernel and user cannot run 64bit kernel
> - Machine has >3GB of RAM
>
> Is there clear evidence that is a common issue? If it is just a "would
> be nice to have in theory" the cost of doing a GFP_DMA32 for i386 would be
> probably not worth doing it. If it's a common issue it might be
> considered, although the "here's a nickle. buy yourself a 64bit CPU"
> strategy would also sound attractive.
>
> Please give a very very strong rationale why you want it. Did you actually
> run into such a situation yourself yet?

Funnily enough I have just today with an app I have which uses 1.2GB
of textures :-)

we are currently using GFP_DMA32 in the TTM allocator code, however
what I really want on x86 non-PAE is GFP_HIGHMEM (as DMA32 does
nothing) however on x86-PAE I don't want that I want a real GFP_DMA32,
and on x86-64 I want the current GFP_DMA32,

Therein lies my problems, the API sucks, but I suppose the DRM TTM is
a special use case (AGP is the same) and I'll accept the fact that can
just make it my own problem, my current solution would be to screw PAE
machines giving them 1GB only and let non-PAE access all 4GB.

Dave.

Re: GFP_DMA32 and PAE x86 machines

[William Lee Irwin III]

On 5/31/07, Andi Kleen <ak@suse.de> wrote:
>> Please give a very very strong rationale why you want it. Did you actually
>> run into such a situation yourself yet?

On Thu, May 31, 2007 at 09:09:58AM +1000, Dave Airlie wrote:
> Funnily enough I have just today with an app I have which uses 1.2GB
> of textures :-)
> we are currently using GFP_DMA32 in the TTM allocator code, however
> what I really want on x86 non-PAE is GFP_HIGHMEM (as DMA32 does
> nothing) however on x86-PAE I don't want that I want a real GFP_DMA32,
> and on x86-64 I want the current GFP_DMA32,
> Therein lies my problems, the API sucks, but I suppose the DRM TTM is
> a special use case (AGP is the same) and I'll accept the fact that can
> just make it my own problem, my current solution would be to screw PAE
> machines giving them 1GB only and let non-PAE access all 4GB.

It's easy enough to spin kernels that allow 4GB-constrained highmem
allocations. The real question is whether it'll be merged.


-- wli

Re: GFP_DMA32 and PAE x86 machines

[Andi Kleen]

> Funnily enough I have just today with an app I have which uses 1.2GB
> of textures :-)

Realistic app which is something users really run?
Also do the other constraints above apply?

If you use pci_alloc_coherent() then it will already take care of the rules
above btw (e.g. DTRT when the machine has <3GB etc.) 

> we are currently using GFP_DMA32 in the TTM allocator code, however
> what I really want on x86 non-PAE is GFP_HIGHMEM (as DMA32 does
> nothing) however on x86-PAE I don't want that I want a real GFP_DMA32,
> and on x86-64 I want the current GFP_DMA32,

You should be probably using pci_alloc_coherent() at least on x86.
It might get more expensive on some obscure platforms (which DRM
likely doesn't support anyways), but if it does perhaps a pci_alloc_noncoherent()
or so would be useful (on x86 it would be the same) 

-Andi