radeon, apertures & memory mapping

radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

Hi !

I'm currently rewriting radeonfb to implement support for dual head, and
ultimately, to make it more friendly to be hooked on DRM for mesa-solo
style setups.

I have some issues however related to the way memory is mapped and
dealing with apertures. Here is the story, suggestions welcome:

The radeon card exposes to the system 2 separate apertures. That is, the
PCI region is actually cut by the hardware in two halves, each of them
beeing an "aperture". Each aperture can have different configuration for
the endian swappers (and possibly the surface tiling registers).

I can configure the apertures to both map to the same bit of video
memory (both covering the framebuffer from 0), or to be "split", that is
aperture 0 covering the framebuffer from 0 to CONFIG_APER_SIZE (size of
an aperture, that is half of the PCI BAR allocation), and aperture 1
covering the framebuffer from CONFIG_APER_SIZE to CONFIG_APER_SIZE*2.

However, I can't change anything to CONFIG_APER_SIZE itself, it's
decided by straps, either HW or in the ROM. So we end up with different
setups depending on how the BIOS has configured things. I know that
Apple chips are usually wired so that CONFIG_APER_SIZE is half the video
memory, so if I use the first mode, I can only access half of the video
RAM from PCI, if I use the second, each aperture maps a different half
of video memory with possibly different endian swapping.

But I think the setups in real life are more diverse and some BIOSes
will have CONFIG_APER_SIZE at least as big as the entire video memory,
thus forcing me to use the "overlapped" setup. In fact, CONFIG_APER_SIZE
may even be smaller than half of the vram and thus limiting the CPU to
part of the VRAM anyway.

I have toyed with all sort of setups, and I have +/- decided to not
bother, and always do this, please tell me what you think:

Always setup HOST_PATH_CNTL.HDP_APER_CNTL to 0. That is, both apertures
are always overlapping. On Macs, or other machines that strap
CONFIG_APER_SIZE to half of VRAM, that means only half of vram can be
directly accessed by the CPU. I think this is fine because of these:

 - We only really need to bother about CPU access for the framebuffer
itself (and possibly the cursor). That is normal non-accelerated fbdev
operations an mmap'ing of the framebuffer in user space. This is not
really a problem if that is limited to some part of vram. It puts a
small constraint on the allocation of video memory: the framebuffer has
to be near the beginning. But my opinion is that a mode switch will
pretty much always invalidate everything that is cached in video memory,
so the whole allocation can be re-done at that point. Things like
texture uploads etc... should use the CP engine and DMA (from either
system or AGP memory).

 - It's actually making things easier for me since I can allocate both
framebuffers next to each other, and probably easier for the future
video memory allocator since it then gets a larger contiguous chunk of
memory to deal with.

 - It's also easier because If I wanted to take advantage of the "split"
setting, I would still need some fallback mecanism for cards who have
the aperture size set to all of vram size.

Note that I also noticed rather inconsistent usage of memory size vs
aperture size in DRI/X. I'm not sure X actually deals with the 2
scenarios here (X always only uses aperture 0 for both heads and changes
the swapper control on each access).

The only drawback is that on the kernel side, for the console, I end up
with the ioremap for the second crtc beeing twice as big as necessary
since I can't "dynamically" ioremap/iounmap on mode switches (that would
be guaranteed to fail after the system has been running for a while). It
would be nice if I could play mapping tricks by doing something like
allocating the 2 bits of virtual space at boot (get_vm_area), and just
remap the pages in them ((un)map_vm_area), but that wouldn't work with
architectures like MIPS who have limitations on where in virtual space
non-cacheable things are.

I could maybe use a single ioremap though, that is use a single
aperture, and then switch the swapper on accesses. Though I should also
be careful not to end up conflicting with a userland process relying on
having the 2 separate aperture swappers stable for the mode on the 2
separate framebuffer mappings... Like X would use fb0 while console
would use fb1 with a different swapper setting. That would blow up for
sure unless fbcon arbitrates accesses with X, which I don't see
happening right away. I suppose we'll have to consider both heads linked
as far as console ownership is concerned, at least for now, until the
kernel console subsystem is overhauled significantly.

Ben.

Re: radeon, apertures & memory mapping

[Jon Smirl]

What about using the overlapped mode and dividing memory into four regions

FB0
PCI visible free mem
FB1
APER_SIZE
non-visible free mem

This way setting the mode on FB0 doesn't always bump into FB1.

The DRM could do this:
FB0
back0
depth0
aux0, etc
PCI visible free mem - textures priority 2
aux1, etc
depth1
back1
FB1
APER_SIZE
non-visible free mem - textures priority 1


On Sun, 13 Mar 2005 12:35:43 +1100, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> I could maybe use a single ioremap though, that is use a single
> aperture, and then switch the swapper on accesses. Though I should also
> be careful not to end up conflicting with a userland process relying on
> having the 2 separate aperture swappers stable for the mode on the 2
> separate framebuffer mappings... Like X would use fb0 while console
> would use fb1 with a different swapper setting. That would blow up for
> sure unless fbcon arbitrates accesses with X, which I don't see
> happening right away. I suppose we'll have to consider both heads linked
> as far as console ownership is concerned, at least for now, until the
> kernel console subsystem is overhauled significantly.

In the long term I was hoping to design thing such that the two heads
can be used by two independent users, each could be running X or
fbdev.

A user space console implementation also makes a lot of sense in the
multiuser case. User space console can be a DRI application instead of
fbdev reducing the need to map.

-- 
Jon Smirl
jonsmirl@gmail.com


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

On Sat, 2005-03-12 at 22:22 -0500, Jon Smirl wrote:
> What about using the overlapped mode and dividing memory into four regions
> 
> FB0
> PCI visible free mem
> FB1
> APER_SIZE
> non-visible free mem
> 
> This way setting the mode on FB0 doesn't always bump into FB1.

I think we don't really have a problem with setting the mode on fb0
possibly bumping into fb1. I mean, setting the mode will require
re-allocation of stuffs in fb memory anyway from what I can tell. I'm
pretty sure than on OS X, applications are notified before an after and
all context stored in fb memory is either lost or backed up into main
memory (though we could be smarter indeed).

The card will be reprogrammed completely, so anybody using it has to be
put on hold, wether it's an fb0 or an fb1 client (and to avoid "issues"
with X among others, I intend to have the setting of MC_FB_LOCATION,
SURFACE registers, etc... to be part of the mode setting, that is). And
finally, I want to blank the screen (using the accel engine) before
setting the new mode, so that we come out "clean" of the mode setting
(without ugly artifact), and I will probably clean both fb's (simpler).

Now there is one reason why putting fb1 on top of the accessible address
space is a good idea though :) It's the ioremap issue. At least, that
way, I can have 2 independant ioremap's of MAX_MAPPED_VRAM from 0 and
from top. If I have fb1 after fb0, since I don't know where fb1 will
actually start, I have no choice but ioremap'ing the second aperture
from the beginning, thus "losing" space.

So I think I'll go your way, it's a good idea, but I'm not 100% sure it
will help much about "not stomping on the other fb". Maybe our allocator
can be smart enough to only invalidate "some" things, or to be able to
move things around (if we have some kind of "indirect" handles to
objects in vram instead of just offsets). 

> The DRM could do this:
> FB0
> back0
> depth0
> aux0, etc
> PCI visible free mem - textures priority 2
> aux1, etc
> depth1
> back1
> FB1
> APER_SIZE
> non-visible free mem - textures priority 1

And AGP memory for textures priority 3 ?

> 
> On Sun, 13 Mar 2005 12:35:43 +1100, Benjamin Herrenschmidt
> <benh@kernel.crashing.org> wrote:
> > I could maybe use a single ioremap though, that is use a single
> > aperture, and then switch the swapper on accesses. Though I should also
> > be careful not to end up conflicting with a userland process relying on
> > having the 2 separate aperture swappers stable for the mode on the 2
> > separate framebuffer mappings... Like X would use fb0 while console
> > would use fb1 with a different swapper setting. That would blow up for
> > sure unless fbcon arbitrates accesses with X, which I don't see
> > happening right away. I suppose we'll have to consider both heads linked
> > as far as console ownership is concerned, at least for now, until the
> > kernel console subsystem is overhauled significantly.
> 
> In the long term I was hoping to design thing such that the two heads
> can be used by two independent users, each could be running X or
> fbdev.

You mean fbcon ? Unless there is proper arbitration, there can't be 2
users. Hopefully we'll get there. But in any way, there can be only one
driver in charge of the "card". That is, if X is using UseFBDev, then
yes, it can use one head and fbcon the other. But if X is reprogramming
the whole card, no way fbdev (and I mean -dev in this case) can use the
other head at the same time, that's just not realistic.

> A user space console implementation also makes a lot of sense in the
> multiuser case. User space console can be a DRI application instead of
> fbdev reducing the need to map.

Sure. Though the mapping done by fbdev stops beeing necessary if we use
only accel ops for everything, including HW cursor. The only issue
becomes implementation of fb_read/fb_write from userspace, but for that,
we can do temporary ioremap's of a few pages, or even use tricks like
locking user pages and doing DMA from them.

Ben.

Re: radeon, apertures & memory mapping

[Jon Smirl]

On Sun, 13 Mar 2005 17:43:09 +1100, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> > In the long term I was hoping to design thing such that the two heads
> > can be used by two independent users, each could be running X or
> > fbdev.
> 
> You mean fbcon ? Unless there is proper arbitration, there can't be 2
> users. Hopefully we'll get there. But in any way, there can be only one
> driver in charge of the "card". That is, if X is using UseFBDev, then
> yes, it can use one head and fbcon the other. But if X is reprogramming
> the whole card, no way fbdev (and I mean -dev in this case) can use the
> other head at the same time, that's just not realistic.

I meant fbcon and XGL. Both will be running on fbdev/DRM and using a
coordinated memory manager, probably in DRM.

Current X is broken for sharing.

-- 
Jon Smirl
jonsmirl@gmail.com

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Michel Dänzer]

On Sun, 2005-03-13 at 17:43 +1100, Benjamin Herrenschmidt wrote:
> 
> And finally, I want to blank the screen (using the accel engine) before
> setting the new mode, so that we come out "clean" of the mode setting
> (without ugly artifact), and I will probably clean both fb's (simpler).

That would break X with UseFBDev.


-- 
Earthling Michel Dänzer      |     Debian (powerpc), X and DRI developer
Libre software enthusiast    |   http://svcs.affero.net/rm.php?r=daenzer

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

On Sun, 2005-03-13 at 23:28 -0500, Michel Dänzer wrote:
> On Sun, 2005-03-13 at 17:43 +1100, Benjamin Herrenschmidt wrote:
> > 
> > And finally, I want to blank the screen (using the accel engine) before
> > setting the new mode, so that we come out "clean" of the mode setting
> > (without ugly artifact), and I will probably clean both fb's (simpler).
> 
> That would break X with UseFBDev.

How so ?

Ben.

Re: radeon, apertures & memory mapping

[Michel Dänzer]

On Mon, 2005-03-14 at 18:12 +1100, Benjamin Herrenschmidt wrote:
> On Sun, 2005-03-13 at 23:28 -0500, Michel Dänzer wrote:
> > On Sun, 2005-03-13 at 17:43 +1100, Benjamin Herrenschmidt wrote:
> > > 
> > > And finally, I want to blank the screen (using the accel engine) before
> > > setting the new mode, so that we come out "clean" of the mode setting
> > > (without ugly artifact), and I will probably clean both fb's (simpler).
> > 
> > That would break X with UseFBDev.
> 
> How so ?

X assumes that the framebuffer (as in video RAM) contents are preserved
on mode switches.


-- 
Earthling Michel Dänzer      |     Debian (powerpc), X and DRI developer
Libre software enthusiast    |   http://svcs.affero.net/rm.php?r=daenzer

Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

On Mon, 2005-03-14 at 11:20 -0500, Michel Dänzer wrote:
> On Mon, 2005-03-14 at 18:12 +1100, Benjamin Herrenschmidt wrote:
> > On Sun, 2005-03-13 at 23:28 -0500, Michel Dänzer wrote:
> > > On Sun, 2005-03-13 at 17:43 +1100, Benjamin Herrenschmidt wrote:
> > > > 
> > > > And finally, I want to blank the screen (using the accel engine) before
> > > > setting the new mode, so that we come out "clean" of the mode setting
> > > > (without ugly artifact), and I will probably clean both fb's (simpler).
> > > 
> > > That would break X with UseFBDev.
> > 
> > How so ?
> 
> X assumes that the framebuffer (as in video RAM) contents are preserved
> on mode switches.

Which I consider bogus :) Oh well... does it do a lot of other crappy
assumptions like that ? What if I just can't allocate it and decide to
put it elsewhere in vram ? (unlikely with my current scheme but
possible). X needs to be fixed.

Ben.

Re: radeon, apertures & memory mapping

[Michel Dänzer]

On Tue, 2005-03-15 at 08:52 +1100, Benjamin Herrenschmidt wrote:
> On Mon, 2005-03-14 at 11:20 -0500, Michel Dänzer wrote:
> > On Mon, 2005-03-14 at 18:12 +1100, Benjamin Herrenschmidt wrote:
> > > On Sun, 2005-03-13 at 23:28 -0500, Michel Dänzer wrote:
> > > > On Sun, 2005-03-13 at 17:43 +1100, Benjamin Herrenschmidt wrote:
> > > > > 
> > > > > And finally, I want to blank the screen (using the accel engine) before
> > > > > setting the new mode, so that we come out "clean" of the mode setting
> > > > > (without ugly artifact), and I will probably clean both fb's (simpler).
> > > > 
> > > > That would break X with UseFBDev.
> > > 
> > > How so ?
> > 
> > X assumes that the framebuffer (as in video RAM) contents are preserved
> > on mode switches.
> 
> Which I consider bogus :) Oh well... does it do a lot of other crappy
> assumptions like that ? What if I just can't allocate it and decide to
> put it elsewhere in vram ? (unlikely with my current scheme but
> possible). X needs to be fixed.

Be that as it may, it remains a fact that such a change would break
existing installations...

I think that mode setting and memory allocation should be separated. X
will always reserve enough video RAM for the largest resolution it uses
for the screen contents.


-- 
Earthling Michel Dänzer      |     Debian (powerpc), X and DRI developer
Libre software enthusiast    |   http://svcs.affero.net/rm.php?r=daenzer

FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

> Be that as it may, it remains a fact that such a change would break
> existing installations...
> 
> I think that mode setting and memory allocation should be separated. X
> will always reserve enough video RAM for the largest resolution it uses
> for the screen contents.

But X has no control on where fbdev will allocate memory. We are
thinking with the "new model" in mind, and so far, a mode setting is
under control of the framebuffer. Content of video memory (framebuffer,
textures, overlay, whatever) simply cannot be considered as preserved
accross mode switches.

We can't also block all evolutions just because we have to support a
broken model. We'll need to find a way to deal with that. An idea would
be for me to do the clearing only when I come from a different bit depth
or from text mode. That is, what i want to avoid, is those artifacts
caused by incorrect bit depth content. A strict mode change isn't an
issue in this case. That would solve my immediate problem.

_BUT_ in the long run, X will have to be smarter. Current UseFBDev or X
"fbdev" can't support dual head properly on top of fbdev anyway, so that
means I have some flexibility. The second head will probably" move on
mode switches since I intend to allocate it at the top of the accessible
aperture as described by Jon.

All of that just keep uncovering more and more issues with our current
fbdev model though. From discussions we had so far, it uncovers the
problem of arbitration. That is, can we simply afford having a process
mmap /dev/fb and blast things to it without any arbitration like we do
today ?

If the answer is yes, then we are in deep trouble for lots of reason:

 - VGA Arbitration might require us to flip/flop MEM/IO enable bits
 - Swapper control as explained earlier unless I can "reserve" a swapper
   for each head, that is manage to have one aperture per head
 - Engine discipline. What if the client on head 0 (like current X) uses
   the engine ? Even if the one on head 1 doesn't, simple framebuffer
   accesses can be enough to lock up the card.
 - etc....

I think that Jon's dream of having totally independant heads that can
run 2 separate applications is a long way away and we have sort-of to
tie /dev/fb's together, though I don't know how to acheive that in
practice, unless we switch to a new API that can enforce it. The current
fbdev API cannot.

DRI can do such things afaik (manage several contexts etc...), at least
provides the infrastructure for it. But until all clients are DRI
clients, we have a problem. That means that any direct fb client has to
take over the entire card. All heads. There is simply no choice, and
that doesn't even help with the VGA arbitration issue which still
require explicit lock/unlock around accesses.

I'm open to suggestions...

Ben.




-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Michel Dänzer]

On Tue, 2005-03-15 at 09:37 +1100, Benjamin Herrenschmidt wrote:
> > Be that as it may, it remains a fact that such a change would break
> > existing installations...
> > 
> > I think that mode setting and memory allocation should be separated. X
> > will always reserve enough video RAM for the largest resolution it uses
> > for the screen contents.
> 
> But X has no control on where fbdev will allocate memory. 

My understanding is that so far, the fbdev API has pretty much implied
that any mode scans out the beginning of the memory accessed via the
framebuffer device, unless the panning ioctl is used. IIRC at least
DirectFB makes basically the same assumptions as X there.

> We are thinking with the "new model" in mind, and so far, a mode setting 
> is under control of the framebuffer. Content of video memory (framebuffer,
> textures, overlay, whatever) simply cannot be considered as preserved
> accross mode switches.
> 
> We can't also block all evolutions just because we have to support a
> broken model. 

I'm not suggesting that, but I do think that tying together mode
switching and memory allocation would be a big mistake.

The EGL API for this is currently being discussed on the dri-egl list
(http://lists.freedesktop.org/mailman/listinfo/dri-egl), you may want to
join in there.

> We'll need to find a way to deal with that. An idea would be for me to 
> do the clearing only when I come from a different bit depth or from text 
> mode. That is, what i want to avoid, is those artifacts caused by 
> incorrect bit depth content. A strict mode change isn't an issue in this 
> case. That would solve my immediate problem.

Sounds good.

> _BUT_ in the long run, X will have to be smarter. Current UseFBDev or X
> "fbdev" can't support dual head properly on top of fbdev anyway, 

Agreed for UseFBDev, but what's the problem with fbdev?


> But until all clients are DRI clients, we have a problem.

Keep in mind that basically the only driver-independent API exposed by
the DRI is OpenGL, so I'm afraid it's unlikely that all fbdev
applications will take that route.


-- 
Earthling Michel Dänzer      |     Debian (powerpc), X and DRI developer
Libre software enthusiast    |   http://svcs.affero.net/rm.php?r=daenzer

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Jon Smirl]

On Mon, 14 Mar 2005 23:59:37 -0500, Michel Dänzer <michel@daenzer.net> wrote:
> On Tue, 2005-03-15 at 09:37 +1100, Benjamin Herrenschmidt wrote:
> > > Be that as it may, it remains a fact that such a change would break
> > > existing installations...
> > >
> > > I think that mode setting and memory allocation should be separated. X
> > > will always reserve enough video RAM for the largest resolution it uses
> > > for the screen contents.
> >
> > But X has no control on where fbdev will allocate memory.
> 
> My understanding is that so far, the fbdev API has pretty much implied
> that any mode scans out the beginning of the memory accessed via the
> framebuffer device, unless the panning ioctl is used. IIRC at least
> DirectFB makes basically the same assumptions as X there.

We are working on adding secondary head support on radeonfb. 
Where does the second buffer go when fbdev is running standalone?

> 
> > We are thinking with the "new model" in mind, and so far, a mode setting
> > is under control of the framebuffer. Content of video memory (framebuffer,
> > textures, overlay, whatever) simply cannot be considered as preserved
> > accross mode switches.
> >
> > We can't also block all evolutions just because we have to support a
> > broken model.
> 
> I'm not suggesting that, but I do think that tying together mode
> switching and memory allocation would be a big mistake.

If fbdev is running standalone and two heads are active, there has to
be some very primitive buffer management going on. This may be as
simple as fb0 starts at 0, and fb1 is at the end of CONFIG_APER_SIZE.

When DRM loads it can install hooks to a more sophisticated memory manager.

> 
> The EGL API for this is currently being discussed on the dri-egl list
> (http://lists.freedesktop.org/mailman/listinfo/dri-egl), you may want to
> join in there.

help me out with the relevant attribute/value pairs for modes

> 
> > We'll need to find a way to deal with that. An idea would be for me to
> > do the clearing only when I come from a different bit depth or from text
> > mode. That is, what i want to avoid, is those artifacts caused by
> > incorrect bit depth content. A strict mode change isn't an issue in this
> > case. That would solve my immediate problem.
> 
> Sounds good.
> 
> > _BUT_ in the long run, X will have to be smarter. Current UseFBDev or X
> > "fbdev" can't support dual head properly on top of fbdev anyway,
> 
> Agreed for UseFBDev, but what's the problem with fbdev?
> 
> 
> > But until all clients are DRI clients, we have a problem.
> 
> Keep in mind that basically the only driver-independent API exposed by
> the DRI is OpenGL, so I'm afraid it's unlikely that all fbdev
> applications will take that route.

There is a 300K OpenGL-ES to framebuffer implementation on sourceforge
that someone might get working in this model.

There is also a big difference between sharing the system with XGL and
an app writen to use fbdev versus supporting an embedded system where
only the fbdev app is running.

> 
> --
> Earthling Michel Dänzer      |     Debian (powerpc), X and DRI developer
> Libre software enthusiast    |   http://svcs.affero.net/rm.php?r=daenzer
> _______________________________________________
> xorg mailing list
> xorg@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/xorg
> 


-- 
Jon Smirl
jonsmirl@gmail.com

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Mon, Mar 14, 2005 at 11:59:37PM -0500, Michel Dänzer wrote:
> On Tue, 2005-03-15 at 09:37 +1100, Benjamin Herrenschmidt wrote:
> > > Be that as it may, it remains a fact that such a change would break
> > > existing installations...
> > > 
> > > I think that mode setting and memory allocation should be separated. X
> > > will always reserve enough video RAM for the largest resolution it uses
> > > for the screen contents.
> > 
> > But X has no control on where fbdev will allocate memory. 
> 
> My understanding is that so far, the fbdev API has pretty much implied
> that any mode scans out the beginning of the memory accessed via the
> framebuffer device, unless the panning ioctl is used. IIRC at least
> DirectFB makes basically the same assumptions as X there.

DirectFB assumes all memory outside var.yres_virtual * fix.line_length is 
preserved. A totally valid assumption in my opinion. It allocates all 
offscreen memory starting from the top of the memory so overlaps with 
fbdev are as rare as possible. Currently it doesn't handle multi head 
except for Matrox G400/G450/G550 TV-out but that is handled without fbdev 
so no API limitations get in the way.

I think that making the assumption that all memory is preserved when the 
memory layout (virtual resolution and depth) doesn't change is perfectly 
valid too. That would allow X to do it's Ctrl-Alt-+ and - things without 
repainting the whole screen.

If radeonfb will allocate the buffer for the second head from the top of 
the memory users would basically have to guess it's location. matroxfb 
simply cuts the memory in two pieces and allocates the buffers from the 
start of each piece. I don't really like that approach. Adding a simple 
byte_offset field to fb_var_screeninfo would solve the problem quite 
nicely but I don't know if such API changes are acceptable at this stage.

> > We are thinking with the "new model" in mind, and so far, a mode setting 
> > is under control of the framebuffer. Content of video memory (framebuffer,
> > textures, overlay, whatever) simply cannot be considered as preserved
> > accross mode switches.
> > 
> > We can't also block all evolutions just because we have to support a
> > broken model. 
> 
> I'm not suggesting that, but I do think that tying together mode
> switching and memory allocation would be a big mistake.

Indeed.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Tue, 2005-03-15 at 08:01 +0200, Ville Syrjälä wrote:

> If radeonfb will allocate the buffer for the second head from the top of 
> the memory users would basically have to guess it's location. matroxfb 
> simply cuts the memory in two pieces and allocates the buffers from the 
> start of each piece. I don't really like that approach. Adding a simple 
> byte_offset field to fb_var_screeninfo would solve the problem quite 
> nicely but I don't know if such API changes are acceptable at this stage.

And we don't know if all HW would support it anyway.

> > > We are thinking with the "new model" in mind, and so far, a mode setting 
> > > is under control of the framebuffer. Content of video memory (framebuffer,
> > > textures, overlay, whatever) simply cannot be considered as preserved
> > > accross mode switches.
> > > 
> > > We can't also block all evolutions just because we have to support a
> > > broken model. 
> > 
> > I'm not suggesting that, but I do think that tying together mode
> > switching and memory allocation would be a big mistake.
> 
> Indeed.

The main issue hwoever, is access arbitration. I'd appreciate your
DirectFB point of view on these things.

Ben.

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Tue, Mar 15, 2005 at 05:59:42PM +1100, Benjamin Herrenschmidt wrote:
> On Tue, 2005-03-15 at 08:01 +0200, Ville Syrjälä wrote:
> 
> > If radeonfb will allocate the buffer for the second head from the top of 
> > the memory users would basically have to guess it's location. matroxfb 
> > simply cuts the memory in two pieces and allocates the buffers from the 
> > start of each piece. I don't really like that approach. Adding a simple 
> > byte_offset field to fb_var_screeninfo would solve the problem quite 
> > nicely but I don't know if such API changes are acceptable at this stage.
> 
> And we don't know if all HW would support it anyway.

Such hardware would be free to ignore any user supplied byte_offset and 
place the buffer anywhere it wants. Even a read-only byte_offset field 
would help. But using the second head would require all apps to be updated 
to be aware of byte_offset :( Maybe some kind of API version thing could 
help here ie. User sets flag X somewhere indicating byte_offset should be 
used instead of changing smem_start.

> > > > We are thinking with the "new model" in mind, and so far, a mode setting 
> > > > is under control of the framebuffer. Content of video memory (framebuffer,
> > > > textures, overlay, whatever) simply cannot be considered as preserved
> > > > accross mode switches.
> > > > 
> > > > We can't also block all evolutions just because we have to support a
> > > > broken model. 
> > > 
> > > I'm not suggesting that, but I do think that tying together mode
> > > switching and memory allocation would be a big mistake.
> > 
> > Indeed.
> 
> The main issue hwoever, is access arbitration. I'd appreciate your
> DirectFB point of view on these things.

DirectFB has it's own asbitration mechanism. It doesn't support using 
multiple framebuffer devices at the same time. For that to work DirectFB 
would just have to know if some of the framebuffer devices are actually 
different outputs of the same card so that it could associate both with 
the same lock and accelerator state.

With the current system I don't see much chance of using accelerated fbcon 
on one head and accelerated DirectFB (or something else) on the other.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Jon Smirl]

On Tue, 15 Mar 2005 16:00:05 +0200, Ville Syrjälä <syrjala@sci.fi> wrote:
> DirectFB has it's own asbitration mechanism. It doesn't support using
> multiple framebuffer devices at the same time. For that to work DirectFB
> would just have to know if some of the framebuffer devices are actually
> different outputs of the same card so that it could associate both with
> the same lock and accelerator state.
> 
> With the current system I don't see much chance of using accelerated fbcon
> on one head and accelerated DirectFB (or something else) on the other.

It looks to be like there needs to be new rules for framebuffer
access. X needs to change, why can't DirectFB change too? This is why
we have so much conflict in graphics. Everyone thinks they completely
own the hardware and can do whatever they want with it. It's obvious
to me, if we add universal aribtration everyone has to change and
follow the new rules.

Aonther approach would be to just say you have to choose to run one of
X, DirectFB, FBUI, XGL and you can't switch between them. Other than
developers I don't know if anyone really runs more than one of these
at a time.

Here's another one:
http://home.comcast.net/~plinius/fbui.html

-- 
Jon Smirl
jonsmirl@gmail.com

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Jan Gukelberger]

On Tue, 2005-03-15 at 09:29 -0500, Jon Smirl wrote:
> Aonther approach would be to just say you have to choose to run one of
> X, DirectFB, FBUI, XGL and you can't switch between them. Other than
> developers I don't know if anyone really runs more than one of these
> at a time.

FWIW, yes we do. 

We're developing a system for neuro-scientific experiments where we
enjoy having a fast and lightweight graphics stack with access to VSYNC
interrupt (DirectFBGL) for reliable frame timing of visual stimuli.
OTOH the experimenter gets a QT control GUI running on a second graphics
card with regular X.

As this is not very common and quite a hassle to setup (e.g. X stomping
on other card's IRQ and the like) I'm eagerly following all efforts to
build a common standard graphics base with improved multi-head and GL
support.

That said, I realise that this is not a very widespread use-case. But
nonetheless there are such applications.

Thanks,
Jan



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
--

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Tue, 2005-03-15 at 16:00 +0200, Ville Syrjälä wrote:

> DirectFB has it's own asbitration mechanism. It doesn't support using 
> multiple framebuffer devices at the same time. For that to work DirectFB 
> would just have to know if some of the framebuffer devices are actually 
> different outputs of the same card so that it could associate both with 
> the same lock and accelerator state.
> 
> With the current system I don't see much chance of using accelerated fbcon 
> on one head and accelerated DirectFB (or something else) on the other.

Oh, I think that won't happen unless directFB maps it's arbitration
mecanism on top of the DRM lock and fbcon does that too.

BTW. Can you quickly explain me the DirectFB arbitration mecanism ?
We'll need to map it on top of the VGA arbiter _at_least_ ...

Ben.



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click

Re: Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Geert Uytterhoeven]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=UTF-8, Size: 2040 bytes --]

On Tue, 15 Mar 2005, Ville [iso-8859-1] Syrjälä wrote:
> I think that making the assumption that all memory is preserved when the 
> memory layout (virtual resolution and depth) doesn't change is perfectly 
> valid too. That would allow X to do it's Ctrl-Alt-+ and - things without 
> repainting the whole screen.

Indeed. The `geometry' part of the screen isn't changed, only the `timings'
part (cfr. the split of fb_var_screeninfo parameters I did in fbutils (which
BTW wasn't ever finished).

> If radeonfb will allocate the buffer for the second head from the top of 
> the memory users would basically have to guess it's location. matroxfb 
> simply cuts the memory in two pieces and allocates the buffers from the 
> start of each piece. I don't really like that approach. Adding a simple 
> byte_offset field to fb_var_screeninfo would solve the problem quite 
> nicely but I don't know if such API changes are acceptable at this stage.

You wouldn't have to guess its location, look at fix.smem_start.

I once did a similar thing for an embedded prototype: take a fixed amount of
memory for both frame buffers (this was a UMA system), fb0 starts from the top,
fb1 starts from the bottom. You can enlarge each frame buffer, until you read
the memory of the other. Each fix.smem_{start,len} corresponds exactly to the
memory allocated to each frame buffer.

Of course, if you also want off-screen memory (i.e. memory beyond
xres_virtual*yres_virtual*bpp/8), things get more complicated, since currently
there's no way for the application to ask for a minimum amount of off-screen
memory. Perhaps a new field in fb_var_screeninfo (and zero means `I don't
care', for backwards compatibility).

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Tue, Mar 15, 2005 at 09:51:40AM +0100, Geert Uytterhoeven wrote:
> On Tue, 15 Mar 2005, Ville [iso-8859-1] Syrj??wrote:
> > If radeonfb will allocate the buffer for the second head from the top of 
> > the memory users would basically have to guess it's location. matroxfb 
> > simply cuts the memory in two pieces and allocates the buffers from the 
> > start of each piece. I don't really like that approach. Adding a simple 
> > byte_offset field to fb_var_screeninfo would solve the problem quite 
> > nicely but I don't know if such API changes are acceptable at this stage.
> 
> You wouldn't have to guess its location, look at fix.smem_start.

But how would someone mmap() the whole memory then? matroxfb already plays 
tricks on fix.smem_start on Millennium I/II cards and it really confuses 
DirectFB's memory allocator.

> I once did a similar thing for an embedded prototype: take a fixed amount of
> memory for both frame buffers (this was a UMA system), fb0 starts from the top,
> fb1 starts from the bottom. You can enlarge each frame buffer, until you read
> the memory of the other. Each fix.smem_{start,len} corresponds exactly to the
> memory allocated to each frame buffer.
> 
> Of course, if you also want off-screen memory (i.e. memory beyond
> xres_virtual*yres_virtual*bpp/8), things get more complicated, since currently
> there's no way for the application to ask for a minimum amount of off-screen
> memory. Perhaps a new field in fb_var_screeninfo (and zero means `I don't
> care', for backwards compatibility).

Offscreen memory is pretty much essential for DirectFB.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

> > I once did a similar thing for an embedded prototype: take a fixed amount of
> > memory for both frame buffers (this was a UMA system), fb0 starts from the top,
> > fb1 starts from the bottom. You can enlarge each frame buffer, until you read
> > the memory of the other. Each fix.smem_{start,len} corresponds exactly to the
> > memory allocated to each frame buffer.
> > 
> > Of course, if you also want off-screen memory (i.e. memory beyond
> > xres_virtual*yres_virtual*bpp/8), things get more complicated, since currently
> > there's no way for the application to ask for a minimum amount of off-screen
> > memory. Perhaps a new field in fb_var_screeninfo (and zero means `I don't
> > care', for backwards compatibility).
> 
> Offscreen memory is pretty much essential for DirectFB.

I still tend to think that offscreen memory is busted on mode
switches...

Now, I agree that cutting the vram in half, and reserving  both halves
to the "offscreen" needs to each head may help avoiding mode switch on
one head busting things used by whoever works on the second head, but
I'm not sure that fits the DRM needs very well..


Ben.




-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Antonino A. Daplas]

On Tuesday 15 March 2005 21:36, Ville Syrjälä wrote:
> On Tue, Mar 15, 2005 at 09:51:40AM +0100, Geert Uytterhoeven wrote:
> > On Tue, 15 Mar 2005, Ville [iso-8859-1] Syrj??wrote:
> > > If radeonfb will allocate the buffer for the second head from the top
> > > of the memory users would basically have to guess it's location.
> > > matroxfb simply cuts the memory in two pieces and allocates the buffers
> > > from the start of each piece. I don't really like that approach. Adding
> > > a simple byte_offset field to fb_var_screeninfo would solve the problem
> > > quite nicely but I don't know if such API changes are acceptable at
> > > this stage.
> >
> > You wouldn't have to guess its location, look at fix.smem_start.
>
> But how would someone mmap() the whole memory then? matroxfb already plays

This is multi-head, right?  That implies one fb per head.  So,  can't you do
separate mmaps?  fb0->fix.smem_start|len and fb1->fix.smem_start|len.

Tony




-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Wed, 2005-03-16 at 07:37 +0800, Antonino A. Daplas wrote:
> On Tuesday 15 March 2005 21:36, Ville Syrjälä wrote:
> > On Tue, Mar 15, 2005 at 09:51:40AM +0100, Geert Uytterhoeven wrote:
> > > On Tue, 15 Mar 2005, Ville [iso-8859-1] Syrj??wrote:
> > > > If radeonfb will allocate the buffer for the second head from the top
> > > > of the memory users would basically have to guess it's location.
> > > > matroxfb simply cuts the memory in two pieces and allocates the buffers
> > > > from the start of each piece. I don't really like that approach. Adding
> > > > a simple byte_offset field to fb_var_screeninfo would solve the problem
> > > > quite nicely but I don't know if such API changes are acceptable at
> > > > this stage.
> > >
> > > You wouldn't have to guess its location, look at fix.smem_start.
> >
> > But how would someone mmap() the whole memory then? matroxfb already plays
> 
> This is multi-head, right?  That implies one fb per head.  So,  can't you do
> separate mmaps?  fb0->fix.smem_start|len and fb1->fix.smem_start|len.

Sure, re-read the thread :) Also, he's worried about management of
offscreen memory. (which is an issue too because of possible problems
with the setup of the apertures -> start of the discussion, and because
it seems that directFB has only been tested on little endian machines
(damn them !) and thus doesn't understand the problem with swapper on
framebuffer access).

Ben.

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Wed, Mar 16, 2005 at 10:50:52AM +1100, Benjamin Herrenschmidt wrote:
> On Wed, 2005-03-16 at 07:37 +0800, Antonino A. Daplas wrote:
> > On Tuesday 15 March 2005 21:36, Ville Syrjälä wrote:
> > > On Tue, Mar 15, 2005 at 09:51:40AM +0100, Geert Uytterhoeven wrote:
> > > > On Tue, 15 Mar 2005, Ville [iso-8859-1] Syrj??wrote:
> > > > > If radeonfb will allocate the buffer for the second head from the top
> > > > > of the memory users would basically have to guess it's location.
> > > > > matroxfb simply cuts the memory in two pieces and allocates the buffers
> > > > > from the start of each piece. I don't really like that approach. Adding
> > > > > a simple byte_offset field to fb_var_screeninfo would solve the problem
> > > > > quite nicely but I don't know if such API changes are acceptable at
> > > > > this stage.
> > > >
> > > > You wouldn't have to guess its location, look at fix.smem_start.
> > >
> > > But how would someone mmap() the whole memory then? matroxfb already plays
> > 
> > This is multi-head, right?  That implies one fb per head.  So,  can't you do
> > separate mmaps?  fb0->fix.smem_start|len and fb1->fix.smem_start|len.
> 
> Sure, re-read the thread :) Also, he's worried about management of
> offscreen memory. (which is an issue too because of possible problems
> with the setup of the apertures -> start of the discussion,

There's also the case with Matrox Millennium I/II cards. They must place 
the visible frame buffer so that no line crosses the boundary of memory 
banks. matroxfb deals with that by moving the buffer and changing 
smem_start and smem_len appropriately. But that is really bad for 
DirectFB's offscreen memory management. After a mode switch the memory 
manager would need to know what kind of initial byte offset was used. Of 
couse it would be possible to determine that from smem_start by knowing 
how the aperture must be aligned. Actually we already do that sort of 
thing to allow hw accelerated rendering when used on matroxfb controlled 
G450/G450/G550 CRTC2. But in that case the offset won't change on mode 
switch.

> and because
> it seems that directFB has only been tested on little endian machines
> (damn them !) and thus doesn't understand the problem with swapper on
> framebuffer access).

Actually people do use it on big-endian systems but since neither the 
mach64, ati128 or radeon drivers play with the swapper settings I can only 
assume that they haven't been tested very extensively.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Wed, 2005-03-16 at 03:47 +0200, Ville Syrjälä wrote:

> There's also the case with Matrox Millennium I/II cards. They must place 
> the visible frame buffer so that no line crosses the boundary of memory 
> banks. matroxfb deals with that by moving the buffer and changing 
> smem_start and smem_len appropriately. But that is really bad for 
> DirectFB's offscreen memory management. After a mode switch the memory 
> manager would need to know what kind of initial byte offset was used. Of 
> couse it would be possible to determine that from smem_start by knowing 
> how the aperture must be aligned. Actually we already do that sort of 
> thing to allow hw accelerated rendering when used on matroxfb controlled 
> G450/G450/G550 CRTC2. But in that case the offset won't change on mode 
> switch.

So it alls end up to -> mode switch has to bust memory layout, and any
assumptions that DirectFB tries to do are incorrect.

The only sane thing that could be done is along the lines you suggested,
that is move most of "fix" into "var" so that the card can tell in
advance what the layout looks like.

But again, this is all going nowhere in the long run. One of the plans
we have is to have the mode setting be independant of the client. While
a client might "lock" it, we want a model where the user triggers a mode
setting via some mode setting & desktop geometry management library that
interfaces to fbdev, and clients be "notified". For that to work, it
also requires proper arbitration though, so we may end up again limit
that functionality to clients using the DRM for artbitration and lock
everything when some guy like DirectFB enters the picture.

> > and because
> > it seems that directFB has only been tested on little endian machines
> > (damn them !) and thus doesn't understand the problem with swapper on
> > framebuffer access).
> 
> Actually people do use it on big-endian systems but since neither the 
> mach64, ati128 or radeon drivers play with the swapper settings I can only 
> assume that they haven't been tested very extensively.

You are wrong here, all of those 3 drivers do play with the swapper
setting, they all 3 set the swapper based on the bit depth of the
screen, so writing an image to offscreen memory with a different bit
depth will be broken. See usage of SURFACE_CNTL in radeonfb for example.

Ben.
  

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Wed, Mar 16, 2005 at 12:51:27PM +1100, Benjamin Herrenschmidt wrote:
> On Wed, 2005-03-16 at 03:47 +0200, Ville Syrjälä wrote:
> 
> > There's also the case with Matrox Millennium I/II cards. They must place 
> > the visible frame buffer so that no line crosses the boundary of memory 
> > banks. matroxfb deals with that by moving the buffer and changing 
> > smem_start and smem_len appropriately. But that is really bad for 
> > DirectFB's offscreen memory management. After a mode switch the memory 
> > manager would need to know what kind of initial byte offset was used. Of 
> > couse it would be possible to determine that from smem_start by knowing 
> > how the aperture must be aligned. Actually we already do that sort of 
> > thing to allow hw accelerated rendering when used on matroxfb controlled 
> > G450/G450/G550 CRTC2. But in that case the offset won't change on mode 
> > switch.
> 
> So it alls end up to -> mode switch has to bust memory layout, and any
> assumptions that DirectFB tries to do are incorrect.

I don't think so. Due to fbdev API limitations DirectFB just can't 
accurately determine how much memory will be used by the fbdev buffer. It 
can make an educated guess though. Just as long as you don't change the 
fact that the fbdev buffer will be located at the beginning of the memory 
that is.

> > > and because
> > > it seems that directFB has only been tested on little endian machines
> > > (damn them !) and thus doesn't understand the problem with swapper on
> > > framebuffer access).
> > 
> > Actually people do use it on big-endian systems but since neither the 
> > mach64, ati128 or radeon drivers play with the swapper settings I can only 
> > assume that they haven't been tested very extensively.
> 
> You are wrong here, all of those 3 drivers do play with the swapper
> setting, they all 3 set the swapper based on the bit depth of the
> screen, so writing an image to offscreen memory with a different bit
> depth will be broken. See usage of SURFACE_CNTL in radeonfb for example.

This was about the DirectFB drivers.

One thing just popped to my head though. If in the future we are going to 
allow graphics cards to render to system memory, using the swapper will no 
longer work. I don't see any other solution that having the CPU perform 
the byte swapping.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Michel Dänzer]

On Wed, 2005-03-16 at 21:51 +0200, Ville Syrjälä wrote:
> 
> One thing just popped to my head though. If in the future we are going to 
> allow graphics cards to render to system memory, using the swapper will no 
> longer work. I don't see any other solution that having the CPU perform 
> the byte swapping.

Sane hardware should have a way to deal with this as well. Either way,
this is hardware specific, so it will probably have to be handled by the
hardware driver(s) somehow.


-- 
Earthling Michel Dänzer      |     Debian (powerpc), X and DRI developer
Libre software enthusiast    |   http://svcs.affero.net/rm.php?r=daenzer

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Wed, Mar 16, 2005 at 04:00:26PM -0500, Michel Dänzer wrote:
> On Wed, 2005-03-16 at 21:51 +0200, Ville Syrjälä wrote:
> > 
> > One thing just popped to my head though. If in the future we are going to 
> > allow graphics cards to render to system memory, using the swapper will no 
> > longer work. I don't see any other solution that having the CPU perform 
> > the byte swapping.
> 
> Sane hardware should have a way to deal with this as well.

In that case I'm not familiar with any sane hardware.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Wed, 2005-03-16 at 16:00 -0500, Michel Dänzer wrote:
> On Wed, 2005-03-16 at 21:51 +0200, Ville Syrjälä wrote:
> > 
> > One thing just popped to my head though. If in the future we are going to 
> > allow graphics cards to render to system memory, using the swapper will no 
> > longer work. I don't see any other solution that having the CPU perform 
> > the byte swapping.
> 
> Sane hardware should have a way to deal with this as well. Either way,
> this is hardware specific, so it will probably have to be handled by the
> hardware driver(s) somehow.

Of course, and radeonfb is what if not a hardware driver ?

Ben.

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Michel Dänzer]

On Thu, 2005-03-17 at 10:28 +1100, Benjamin Herrenschmidt wrote:
> On Wed, 2005-03-16 at 16:00 -0500, Michel D채nzer wrote:
> > On Wed, 2005-03-16 at 21:51 +0200, Ville Syrj채l채 wrote:
> > > 
> > > One thing just popped to my head though. If in the future we are going to 
> > > allow graphics cards to render to system memory, using the swapper will no 
> > > longer work. I don't see any other solution that having the CPU perform 
> > > the byte swapping.
> > 
> > Sane hardware should have a way to deal with this as well. Either way,
> > this is hardware specific, so it will probably have to be handled by the
> > hardware driver(s) somehow.
> 
> Of course, and radeonfb is what if not a hardware driver ?

Who said it was anything else? Is radeonfb gonna handle the offscreen
surfaces though? My point was that the hardware driver(s) should be
involved in the decision on what format/byte order/... should be used
for a surface, instead of just hardcoding fixed formats and having the
CPU do possibly unnecessary byte swapping.


-- 
Earthling Michel D채nzer      |     Debian (powerpc), X and DRI developer
Libre software enthusiast    |   http://svcs.affero.net/rm.php?r=daenzer

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Wed, 2005-03-16 at 21:51 +0200, Ville Syrjälä wrote:

> This was about the DirectFB drivers.
> 
> One thing just popped to my head though. If in the future we are going to 
> allow graphics cards to render to system memory, using the swapper will no 
> longer work. I don't see any other solution that having the CPU perform 
> the byte swapping.

There is a _lot_ of existing code that will stop working, MOL is an
example, it requires proper "native" format for the framebuffer. X too
in it's current incarnation (fb format is defined at compile time I
think).

So the swapper is _needed_ for framebuffer access.

The best we can do is save/change/restore it around accesses like we do
already when copying YUV frames to video memory.

Now regarding the setup of the apertures, I suppose that I'll do
something complicated that nobody will use right, that is use 2 separate
apertures contiguous with each framebuffer at the beginning of each
aperture when CONFIG_APER_SIZE is 1/2 of vram size, and 2 separate
apertures contiguous with each framebuffer together in the first one in
the other cases. Only difference: first case will allow macs to have
separate swappers for each aperture. All other cases will have both
swappers set the same way. Hopefully, all "Mac" cards have
CONFIG_APER_SIZE set to half the vram.

Ben.

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Michel Dänzer]

On Wed, 2005-03-16 at 10:33 +1100, Benjamin Herrenschmidt wrote:
> 
> Now, I agree that cutting the vram in half, and reserving  both halves
> to the "offscreen" needs to each head may help avoiding mode switch on
> one head busting things used by whoever works on the second head, but
> I'm not sure that fits the DRM needs very well..

It probably doesn't, but it seems to me like that's all that can be done
with the current fbdev API.


On Wed, 2005-03-16 at 12:51 +1100, Benjamin Herrenschmidt wrote: 
> On Wed, 2005-03-16 at 03:47 +0200, Ville Syrjälä wrote:
> 
> > There's also the case with Matrox Millennium I/II cards. They must place 
> > the visible frame buffer so that no line crosses the boundary of memory 
> > banks. [...]

That still doesn't require any changes on mode switches as long as the
pitch stays the same, does it?

> So it alls end up to -> mode switch has to bust memory layout, [...]

Why? All that a mode switch changes is how the current screen surface is
scanned out. It shouldn't have any implicit effect on memory allocation
or even contents.


> [...] For that to work, it also requires proper arbitration though, so we 
> may end up again limit that functionality to clients using the DRM for 
> artbitration and lock everything when some guy like DirectFB enters the 
> picture.

I was thinking about something like that as well, may be the only way to
preserve the current fbdev API and offer something more sophisticated.


> > > and because
> > > it seems that directFB has only been tested on little endian machines
> > > (damn them !) and thus doesn't understand the problem with swapper on
> > > framebuffer access).
> > 
> > Actually people do use it on big-endian systems but since neither the 
> > mach64, ati128 or radeon drivers play with the swapper settings I can only 
> > assume that they haven't been tested very extensively.
> 
> You are wrong here, all of those 3 drivers do play with the swapper
> setting, [...]

I think he was referring to the DirectFB drivers.


-- 
Earthling Michel Dänzer      |     Debian (powerpc), X and DRI developer
Libre software enthusiast    |   http://svcs.affero.net/rm.php?r=daenzer

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Wed, 2005-03-16 at 00:21 -0500, Michel Dänzer wrote:

> > > 
> > > Actually people do use it on big-endian systems but since neither the 
> > > mach64, ati128 or radeon drivers play with the swapper settings I can only 
> > > assume that they haven't been tested very extensively.
> > 
> > You are wrong here, all of those 3 drivers do play with the swapper
> > setting, [...]
> 
> I think he was referring to the DirectFB drivers.

Well, do they "revert" it after atyfb/aty128fb/radeonfb set it then ?
it's definitely set on mode switch.

Ben.

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Wed, Mar 16, 2005 at 05:31:16PM +1100, Benjamin Herrenschmidt wrote:
> On Wed, 2005-03-16 at 00:21 -0500, Michel Dänzer wrote:
> 
> > > > 
> > > > Actually people do use it on big-endian systems but since neither the 
> > > > mach64, ati128 or radeon drivers play with the swapper settings I can only 
> > > > assume that they haven't been tested very extensively.
> > > 
> > > You are wrong here, all of those 3 drivers do play with the swapper
> > > setting, [...]
> > 
> > I think he was referring to the DirectFB drivers.

Exactly. I prorably should have mentioned that explicitly.

> Well, do they "revert" it after atyfb/aty128fb/radeonfb set it then ?
> it's definitely set on mode switch.

The point is that there can be offscreen surfaces with different depth 
than the fbdev surface.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Adam Jackson]

[-- Attachment #1.1: Type: text/plain, Size: 439 bytes --]

On Wednesday 16 March 2005 09:09, Ville Syrjälä wrote:
> On Wed, Mar 16, 2005 at 05:31:16PM +1100, Benjamin Herrenschmidt wrote:
> > Well, do they "revert" it after atyfb/aty128fb/radeonfb set it then ?
> > it's definitely set on mode switch.
>
> The point is that there can be offscreen surfaces with different depth
> than the fbdev surface.

_Will_ be.  Otherwise GL and Render performance drops through the floor.

- ajax

[-- Attachment #1.2: Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 144 bytes --]

_______________________________________________
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Wed, 2005-03-16 at 11:48 -0500, Adam Jackson wrote:
> On Wednesday 16 March 2005 09:09, Ville Syrjälä wrote:
> > On Wed, Mar 16, 2005 at 05:31:16PM +1100, Benjamin Herrenschmidt wrote:
> > > Well, do they "revert" it after atyfb/aty128fb/radeonfb set it then ?
> > > it's definitely set on mode switch.
> >
> > The point is that there can be offscreen surfaces with different depth
> > than the fbdev surface.
> 
> _Will_ be.  Otherwise GL and Render performance drops through the floor.

Yes, but again, GL/DRI knows how to play with the swapper, that is not
my point. Looks like people here prefer arguing for the sake of it
rather than trying to make progress...

I think I'll just decide something randomly in radeonfb and let folks
cope with it.

Ben.

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Roland Scheidegger]

Ville Syrjälä wrote:
  > I think that making the assumption that all memory is preserved when 
the
> memory layout (virtual resolution and depth) doesn't change is perfectly 
> valid too. That would allow X to do it's Ctrl-Alt-+ and - things without 
> repainting the whole screen.
I'm not sure I agree here, as it's not always true. For instance, the 
radeon has some restrictions whether it can use tiling or not with a 
certain mode (interlace/double scan) thus you need to redraw everything 
anyway (which is exactly why I implemented a driver workaround to 
repaint everything when that happens - in fact the workaround also gets 
rid of the offscreen contents, which is not necessary, but was much 
easier to implement, since I couldn't find an easy way to "invalidate 
the framebuffer"). What's the big deal with repainting everything? It's 
not like you would do 100 mode changes per second so it would be 
performance-critical...

Roland

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Tue, Mar 15, 2005 at 12:30:56PM +0100, Roland Scheidegger wrote:
> Ville Syrjälä wrote:
>  > I think that making the assumption that all memory is preserved when 
> the
> >memory layout (virtual resolution and depth) doesn't change is perfectly 
> >valid too. That would allow X to do it's Ctrl-Alt-+ and - things without 
> >repainting the whole screen.
> I'm not sure I agree here, as it's not always true. For instance, the 
> radeon has some restrictions whether it can use tiling or not with a 
> certain mode (interlace/double scan) thus you need to redraw everything 
> anyway

I didn't know about that. My first thought would be to disallow such modes 
but knowing that X lacks a proper fullscreen API that might not be a 
realistic option.

> (which is exactly why I implemented a driver workaround to 
> repaint everything when that happens - in fact the workaround also gets 
> rid of the offscreen contents, which is not necessary, but was much 
> easier to implement, since I couldn't find an easy way to "invalidate 
> the framebuffer"). What's the big deal with repainting everything? It's 
> not like you would do 100 mode changes per second so it would be 
> performance-critical...

I don't really have a big deal with invalidating the visible part of the 
memory.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Michel Dänzer]

On Tue, 2005-03-15 at 12:30 +0100, Roland Scheidegger wrote:
> Ville Syrjälä wrote:
>   > I think that making the assumption that all memory is preserved when 
> the
> > memory layout (virtual resolution and depth) doesn't change is perfectly 
> > valid too. That would allow X to do it's Ctrl-Alt-+ and - things without 
> > repainting the whole screen.
> I'm not sure I agree here, as it's not always true. For instance, the 
> radeon has some restrictions whether it can use tiling or not with a 
> certain mode (interlace/double scan) thus you need to redraw everything 
> anyway (which is exactly why I implemented a driver workaround to 
> repaint everything when that happens - in fact the workaround also gets 
> rid of the offscreen contents, which is not necessary, but was much 
> easier to implement, since I couldn't find an easy way to "invalidate 
> the framebuffer"). What's the big deal with repainting everything? It's 
> not like you would do 100 mode changes per second so it would be 
> performance-critical...

It's ugly, but that's not the point. The point is that all deployed
versions of X (and even current X.org CVS head still, in fact) make this
assumption.


-- 
Earthling Michel Dänzer      |     Debian (powerpc), X and DRI developer
Libre software enthusiast    |   http://svcs.affero.net/rm.php?r=daenzer

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

> It's ugly, but that's not the point. The point is that all deployed
> versions of X (and even current X.org CVS head still, in fact) make this
> assumption.

Oh, that's fine and that's not a problem. I will only repaint the
framebuffer on bit depth or line lenght changes. I'm trying to talk
about the _future_ here. That is support for dual head at the fbdev
level and other niceties.

We simply cannot guarantee preserving the video memory accross mode
switches. We have enumerated enough examples of that, and Ville himself
came up with a nice one about Matrox.

What we _can_ do is let people know what was expelled from video memory
eventually. But even the "let's ask fdbev what will change" before the
actual mode switch isn't really a good idea in the long run since it
sort-of defeats the purpose of having a common memory manager.

Ben.

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Wed, Mar 16, 2005 at 02:09:18PM +1100, Benjamin Herrenschmidt wrote:
> 
> > It's ugly, but that's not the point. The point is that all deployed
> > versions of X (and even current X.org CVS head still, in fact) make this
> > assumption.
> 
> Oh, that's fine and that's not a problem. I will only repaint the
> framebuffer on bit depth or line lenght changes. I'm trying to talk
> about the _future_ here. That is support for dual head at the fbdev
> level and other niceties.

I don't see the current system slowly evolving into some superb future 
system with an in kernel memory manager. The current APIs just have too 
many limitations. I think the memory manager must be the foundation of 
everything and after it's in place the fbdev API should be able to use it. 
The only change to simple fbdev apps would be that they can't get access 
to any offscreen memory as they do now. Something like DirectFB would need 
to change to accomodate the new system but I don't see that as a problem.

I think the best short term option for radeonfb is to simply follow 
matroxfb's example and cut the memory into two parts. The cutoff point 
should probably be configurable via a module option.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Michel Dänzer]

On Wed, 2005-03-16 at 22:08 +0200, Ville Syrjälä wrote:
> 
> I don't see the current system slowly evolving into some superb future 
> system with an in kernel memory manager. The current APIs just have too 
> many limitations. I think the memory manager must be the foundation of 
> everything and after it's in place the fbdev API should be able to use it. 
> The only change to simple fbdev apps would be that they can't get access 
> to any offscreen memory as they do now. Something like DirectFB would need 
> to change to accomodate the new system but I don't see that as a problem.

I agree on this.


-- 
Earthling Michel Dänzer      |     Debian (powerpc), X and DRI developer
Libre software enthusiast    |   http://svcs.affero.net/rm.php?r=daenzer

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Wed, 2005-03-16 at 15:42 -0500, Michel Dänzer wrote:
> On Wed, 2005-03-16 at 22:08 +0200, Ville Syrjälä wrote:
> > 
> > I don't see the current system slowly evolving into some superb future 
> > system with an in kernel memory manager. The current APIs just have too 
> > many limitations. I think the memory manager must be the foundation of 
> > everything and after it's in place the fbdev API should be able to use it. 
> > The only change to simple fbdev apps would be that they can't get access 
> > to any offscreen memory as they do now. Something like DirectFB would need 
> > to change to accomodate the new system but I don't see that as a problem.
> 
> I agree on this.

Ok, ok, I'll do crap since that's what everybody wants.

Ben.

Re: Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Jon Smirl]

On Thu, 17 Mar 2005 10:26:57 +1100, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> On Wed, 2005-03-16 at 15:42 -0500, Michel Dänzer wrote:
> > On Wed, 2005-03-16 at 22:08 +0200, Ville Syrjälä wrote:
> > >
> > > I don't see the current system slowly evolving into some superb future
> > > system with an in kernel memory manager. The current APIs just have too
> > > many limitations. I think the memory manager must be the foundation of
> > > everything and after it's in place the fbdev API should be able to use it.
> > > The only change to simple fbdev apps would be that they can't get access
> > > to any offscreen memory as they do now. Something like DirectFB would need
> > > to change to accomodate the new system but I don't see that as a problem.
> >
> > I agree on this.
> 
> Ok, ok, I'll do crap since that's what everybody wants.

Why don't we modify the new radoenfb driver to have a real
arbitration/management layer. Directfb can continue to use the old
driver. The rule is that you can't mix old and new style fbdev drivers
in a system. Over time DirectFb and X can be fixed to use the new
model.

There is going to have to be a transition path while we debug the
arbitration/management layer and figure out the right API for it.
Supporting multiple disparate users of the graphics hardware is a
quite complex feature that no other operating system implements.


> 
> Ben.
> 
> 
> _______________________________________________
> xorg mailing list
> xorg@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/xorg
> 


-- 
Jon Smirl
jonsmirl@gmail.com


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

> Why don't we modify the new radoenfb driver to have a real
> arbitration/management layer. Directfb can continue to use the old
> driver. The rule is that you can't mix old and new style fbdev drivers
> in a system. Over time DirectFb and X can be fixed to use the new
> model.
> 
> There is going to have to be a transition path while we debug the
> arbitration/management layer and figure out the right API for it.
> Supporting multiple disparate users of the graphics hardware is a
> quite complex feature that no other operating system implements.

I prefer letting the heat cool down for now. I'll implement the actual
mode setting etc... with a very basic memory management, and we'll move
from there once I have the code.

Ben.

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Alex Deucher]

On Wed, 16 Mar 2005 22:08:08 +0200, Ville Syrjälä <syrjala@sci.fi> wrote:
> On Wed, Mar 16, 2005 at 02:09:18PM +1100, Benjamin Herrenschmidt wrote:
> >
> > > It's ugly, but that's not the point. The point is that all deployed
> > > versions of X (and even current X.org CVS head still, in fact) make this
> > > assumption.
> >
> > Oh, that's fine and that's not a problem. I will only repaint the
> > framebuffer on bit depth or line lenght changes. I'm trying to talk
> > about the _future_ here. That is support for dual head at the fbdev
> > level and other niceties.
> 
> I don't see the current system slowly evolving into some superb future
> system with an in kernel memory manager. The current APIs just have too
> many limitations. I think the memory manager must be the foundation of
> everything and after it's in place the fbdev API should be able to use it.
> The only change to simple fbdev apps would be that they can't get access
> to any offscreen memory as they do now. Something like DirectFB would need
> to change to accomodate the new system but I don't see that as a problem.
> 
> I think the best short term option for radeonfb is to simply follow
> matroxfb's example and cut the memory into two parts. The cutoff point
> should probably be configurable via a module option.
> 

if we are going to go through the trouble to do it at all why not do
it "the right way"?  We could also work on updating the fbdev drivers
to the "new version" out of tree until they have stabilized and then
we can work on merging them into the kernel.  that should give
GUI/driver developers time to adapt their respecitve projects.

Alex

> --
> Ville Syrjälä
> syrjala@sci.fi
> http://www.sci.fi/~syrjala/

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Wed, Mar 16, 2005 at 03:58:07PM -0500, Alex Deucher wrote:
> On Wed, 16 Mar 2005 22:08:08 +0200, Ville Syrjälä <syrjala@sci.fi> wrote:
> > On Wed, Mar 16, 2005 at 02:09:18PM +1100, Benjamin Herrenschmidt wrote:
> > >
> > > > It's ugly, but that's not the point. The point is that all deployed
> > > > versions of X (and even current X.org CVS head still, in fact) make this
> > > > assumption.
> > >
> > > Oh, that's fine and that's not a problem. I will only repaint the
> > > framebuffer on bit depth or line lenght changes. I'm trying to talk
> > > about the _future_ here. That is support for dual head at the fbdev
> > > level and other niceties.
> > 
> > I don't see the current system slowly evolving into some superb future
> > system with an in kernel memory manager. The current APIs just have too
> > many limitations. I think the memory manager must be the foundation of
> > everything and after it's in place the fbdev API should be able to use it.
> > The only change to simple fbdev apps would be that they can't get access
> > to any offscreen memory as they do now. Something like DirectFB would need
> > to change to accomodate the new system but I don't see that as a problem.
> > 
> > I think the best short term option for radeonfb is to simply follow
> > matroxfb's example and cut the memory into two parts. The cutoff point
> > should probably be configurable via a module option.
> > 
> 
> if we are going to go through the trouble to do it at all why not do
> it "the right way"?

I haven't seen anyone coming forward with a design/code for the memory 
manager.

In the meantime I'm assuming that people might want to make some use of 
their dualhead cards...

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Alex Deucher]

On Wed, 16 Mar 2005 23:08:12 +0200, Ville Syrjälä <syrjala@sci.fi> wrote:
> On Wed, Mar 16, 2005 at 03:58:07PM -0500, Alex Deucher wrote:
> > On Wed, 16 Mar 2005 22:08:08 +0200, Ville Syrjälä <syrjala@sci.fi> wrote:
> > > On Wed, Mar 16, 2005 at 02:09:18PM +1100, Benjamin Herrenschmidt wrote:
> > > >
> > > > > It's ugly, but that's not the point. The point is that all deployed
> > > > > versions of X (and even current X.org CVS head still, in fact) make this
> > > > > assumption.
> > > >
> > > > Oh, that's fine and that's not a problem. I will only repaint the
> > > > framebuffer on bit depth or line lenght changes. I'm trying to talk
> > > > about the _future_ here. That is support for dual head at the fbdev
> > > > level and other niceties.
> > >
> > > I don't see the current system slowly evolving into some superb future
> > > system with an in kernel memory manager. The current APIs just have too
> > > many limitations. I think the memory manager must be the foundation of
> > > everything and after it's in place the fbdev API should be able to use it.
> > > The only change to simple fbdev apps would be that they can't get access
> > > to any offscreen memory as they do now. Something like DirectFB would need
> > > to change to accomodate the new system but I don't see that as a problem.
> > >
> > > I think the best short term option for radeonfb is to simply follow
> > > matroxfb's example and cut the memory into two parts. The cutoff point
> > > should probably be configurable via a module option.
> > >
> >
> > if we are going to go through the trouble to do it at all why not do
> > it "the right way"?
> 
> I haven't seen anyone coming forward with a design/code for the memory
> manager.
> 
> In the meantime I'm assuming that people might want to make some use of
> their dualhead cards...
> 

Good point.

> --
> Ville Syrjälä
> syrjala@sci.fi
> http://www.sci.fi/~syrjala/
>

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Wed, 2005-03-16 at 23:08 +0200, Ville Syrjälä wrote:
> On Wed, Mar 16, 2005 at 03:58:07PM -0500, Alex Deucher wrote:

> I haven't seen anyone coming forward with a design/code for the memory 
> manager.
> 
> In the meantime I'm assuming that people might want to make some use of 
> their dualhead cards...

Are you aware that with the current fbdev API, there will simply be no
working use of dual head ? As soon as somebody will try to do 2
different things on the 2 heads, it will either lockup due to lack of
engine arbitration, or have wrong endianness, or whatever ...

Ben.

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Thu, Mar 17, 2005 at 10:30:01AM +1100, Benjamin Herrenschmidt wrote:
> On Wed, 2005-03-16 at 23:08 +0200, Ville Syrjälä wrote:
> > On Wed, Mar 16, 2005 at 03:58:07PM -0500, Alex Deucher wrote:
> 
> > I haven't seen anyone coming forward with a design/code for the memory 
> > manager.
> > 
> > In the meantime I'm assuming that people might want to make some use of 
> > their dualhead cards...
> 
> Are you aware that with the current fbdev API, there will simply be no
> working use of dual head ? As soon as somebody will try to do 2
> different things on the 2 heads, it will either lockup due to lack of
> engine arbitration, or have wrong endianness, or whatever ...

I understand you can't have userspace program the accelerator while 
someone else is doing the same thing. Oh and I now understand that the 
same really applies to direct framebuffer access due to the swapper. I 
hadn't really thought about that issue before since I don't own a 
big-endian system. I really must try to get one...

So basically to fix both issues we need some locks everyone must acquire 
before accessing the hardware.

With the current "mmap() registers and go" interface the accelerator lock 
wouldn't actually guarantee anything but it would allow well behaving 
applications to share the accelerator. Good behaviour is already expected 
from the applications anyway due to the direct access to hardware 
registers.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

> I understand you can't have userspace program the accelerator while 
> someone else is doing the same thing. Oh and I now understand that the 
> same really applies to direct framebuffer access due to the swapper.  

And you can't have someone program the accelerator while somebody does
direct access neither. It's basically all exclusive.
> I 
> hadn't really thought about that issue before since I don't own a 
> big-endian system. I really must try to get one...

The only thing that would "work" without arbitration is direct
famebuffer access on both, with both using a different swapper, that is
a different aperture. That is either doing the the way I described
earlier (having both apertures overlap the beginning of memory) or doing
the mapping the "right" way (that is both apertures mapping half of
memory so that together they map all of it).

The problem is that I think a bunch of BIOSes for x86 didn't care since
they don't use the swapper and didn't set the aperture size properly.

> So basically to fix both issues we need some locks everyone must acquire 
> before accessing the hardware.

Plus the VGA arbitration of course ... But yes. That's more or less what
DRM provides.

> With the current "mmap() registers and go" interface the accelerator lock 
> wouldn't actually guarantee anything but it would allow well behaving 
> applications to share the accelerator. Good behaviour is already expected 
> from the applications anyway due to the direct access to hardware 
> registers.

It's more than the accelerator. A framebuffer access concurrent with
accelerator operations can lockup too.

Ben.

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Thu, Mar 17, 2005 at 12:12:58PM +1100, Benjamin Herrenschmidt wrote:
> 
> > I understand you can't have userspace program the accelerator while 
> > someone else is doing the same thing. Oh and I now understand that the 
> > same really applies to direct framebuffer access due to the swapper.  
> 
> And you can't have someone program the accelerator while somebody does
> direct access neither. It's basically all exclusive.

I haven't seen that happen on any hardware I own. Matrox specs explicitly 
mention that there is no need to synchronize accelerator and direct 
framebuffer access.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Thu, 2005-03-17 at 03:25 +0200, Ville Syrjälä wrote:
> On Thu, Mar 17, 2005 at 12:12:58PM +1100, Benjamin Herrenschmidt wrote:
> > 
> > > I understand you can't have userspace program the accelerator while 
> > > someone else is doing the same thing. Oh and I now understand that the 
> > > same really applies to direct framebuffer access due to the swapper.  
> > 
> > And you can't have someone program the accelerator while somebody does
> > direct access neither. It's basically all exclusive.
> 
> I haven't seen that happen on any hardware I own. Matrox specs explicitly 
> mention that there is no need to synchronize accelerator and direct 
> framebuffer access.

You are lucky then :)

Ben.

Re: Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Geert Uytterhoeven]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=UTF-8, Size: 1139 bytes --]

On Thu, 17 Mar 2005, Ville [iso-8859-1] Syrjälä wrote:
> On Thu, Mar 17, 2005 at 12:12:58PM +1100, Benjamin Herrenschmidt wrote:
> > > I understand you can't have userspace program the accelerator while 
> > > someone else is doing the same thing. Oh and I now understand that the 
> > > same really applies to direct framebuffer access due to the swapper.  
> > 
> > And you can't have someone program the accelerator while somebody does
> > direct access neither. It's basically all exclusive.
> 
> I haven't seen that happen on any hardware I own. Matrox specs explicitly 
> mention that there is no need to synchronize accelerator and direct 
> framebuffer access.

Really? I was always given Matrox as an example of a card that would lock up if
you access the frame buffer while the accelerator is busy...

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: [Linux-fbdev-devel] Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Thu, Mar 17, 2005 at 10:08:15AM +0100, Geert Uytterhoeven wrote:
> On Thu, 17 Mar 2005, Ville [iso-8859-1] Syrj??wrote:
> > On Thu, Mar 17, 2005 at 12:12:58PM +1100, Benjamin Herrenschmidt wrote:
> > > > I understand you can't have userspace program the accelerator while 
> > > > someone else is doing the same thing. Oh and I now understand that the 
> > > > same really applies to direct framebuffer access due to the swapper.  
> > > 
> > > And you can't have someone program the accelerator while somebody does
> > > direct access neither. It's basically all exclusive.
> > 
> > I haven't seen that happen on any hardware I own. Matrox specs explicitly 
> > mention that there is no need to synchronize accelerator and direct 
> > framebuffer access.
> 
> Really?

Really.

> I was always given Matrox as an example of a card that would lock up if
> you access the frame buffer while the accelerator is busy...

Apparently they didn't know what they were talking about.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

> DirectFB assumes all memory outside var.yres_virtual * fix.line_length is 
> preserved. A totally valid assumption in my opinion. 

Except that you can't know in advance how much fix.line_length will be.
The "fix" isn't really "fixed". Different cards will have different
requirements depending on the bit depth for example. On radeonfb, the
line_length will vary due to alignment constraints related to the
engine, or due to tiling, etc.... etc...

So you basically don't know in advance what will be preserved... (And
you can't, unless you start having all sort of card specific knowledge).

 
Ben.

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Wed, Mar 16, 2005 at 09:44:19AM +1100, Benjamin Herrenschmidt wrote:
> 
> > DirectFB assumes all memory outside var.yres_virtual * fix.line_length is 
> > preserved. A totally valid assumption in my opinion. 
> 
> Except that you can't know in advance how much fix.line_length will be.
> The "fix" isn't really "fixed". Different cards will have different
> requirements depending on the bit depth for example. On radeonfb, the
> line_length will vary due to alignment constraints related to the
> engine, or due to tiling, etc.... etc...
> 
> So you basically don't know in advance what will be preserved... (And
> you can't, unless you start having all sort of card specific knowledge).

True. Currently DirectFB doesn't handle this correctly. But that could be 
easily fixed if only line_length wasn't totally misplaced. It really 
belongs to fb_var_screeninfo. We could first test the mode with 
FB_ACTIVATE_TEST and actually see how much memory it needs and could 
evict enough offscreen surfaces to make room before actually setting the 
mode. Currently it would need some guesswork.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Wed, 2005-03-16 at 01:05 +0200, Ville Syrjälä wrote:

> True. Currently DirectFB doesn't handle this correctly. But that could be 
> easily fixed if only line_length wasn't totally misplaced. It really 
> belongs to fb_var_screeninfo. We could first test the mode with 
> FB_ACTIVATE_TEST and actually see how much memory it needs and could 
> evict enough offscreen surfaces to make room before actually setting the 
> mode. Currently it would need some guesswork.

Agreed. line_lenght was misplaced... I suppose there may be interest in
making a v2 of the var structure and using a flag to indicate wether we
are passing a v1 or a v2 one...

In the meantime, can you tell me more about your arbitration scheme ?
Arbitration is what I'm trying to solve. There are two main issues:
multiple card arbitration (that is, the VGA cruft to deal with) and
arbitration between access on X heads of the same card (which would be
nice, but not mandatory, DirectFB may take over all heads of the card,
doing that wuld require proper use of the DRM lock & engine context
switch, more of a long term goal).

Ben.

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Wed, Mar 16, 2005 at 10:49:11AM +1100, Benjamin Herrenschmidt wrote:
> In the meantime, can you tell me more about your arbitration scheme ?

There is a lock associated with the graphics card. The lock is always 
taken before programming the hardware. Other things wanting access to the 
hardware wait until the lock is released.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Wed, 2005-03-16 at 22:55 +0200, Ville Syrjälä wrote:
> On Wed, Mar 16, 2005 at 10:49:11AM +1100, Benjamin Herrenschmidt wrote:
> > In the meantime, can you tell me more about your arbitration scheme ?
> 
> There is a lock associated with the graphics card. The lock is always 
> taken before programming the hardware. Other things wanting access to the 
> hardware wait until the lock is released.

Ok, so it would be easy to have directFB use an external arbiter without
breaking existing clients ? It will need at least to use the vga arbiter
that I'm about to finish, that should allow at least to have X on one
card and directFB on another without conflict.

Ben.

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ville Syrjälä]

On Thu, Mar 17, 2005 at 10:27:56AM +1100, Benjamin Herrenschmidt wrote:
> On Wed, 2005-03-16 at 22:55 +0200, Ville Syrjälä wrote:
> > On Wed, Mar 16, 2005 at 10:49:11AM +1100, Benjamin Herrenschmidt wrote:
> > > In the meantime, can you tell me more about your arbitration scheme ?
> > 
> > There is a lock associated with the graphics card. The lock is always 
> > taken before programming the hardware. Other things wanting access to the 
> > hardware wait until the lock is released.
> 
> Ok, so it would be easy to have directFB use an external arbiter without
> breaking existing clients ? It will need at least to use the vga arbiter
> that I'm about to finish, that should allow at least to have X on one
> card and directFB on another without conflict.

Is the vga arbiter required for something else besides access to some 
legacy ports? DirectFB only uses legacy ports to wait for vsync if a 
better method is not available.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Thu, 2005-03-17 at 02:14 +0200, Ville Syrjälä wrote:
> On Thu, Mar 17, 2005 at 10:27:56AM +1100, Benjamin Herrenschmidt wrote:
> > On Wed, 2005-03-16 at 22:55 +0200, Ville Syrjälä wrote:
> > > On Wed, Mar 16, 2005 at 10:49:11AM +1100, Benjamin Herrenschmidt wrote:
> > > > In the meantime, can you tell me more about your arbitration scheme ?
> > > 
> > > There is a lock associated with the graphics card. The lock is always 
> > > taken before programming the hardware. Other things wanting access to the 
> > > hardware wait until the lock is released.
> > 
> > Ok, so it would be easy to have directFB use an external arbiter without
> > breaking existing clients ? It will need at least to use the vga arbiter
> > that I'm about to finish, that should allow at least to have X on one
> > card and directFB on another without conflict.
> 
> Is the vga arbiter required for something else besides access to some 
> legacy ports? DirectFB only uses legacy ports to wait for vsync if a 
> better method is not available.

Yes. I need to extend the Arbiter API a bit in fact to deal with clients
that don't "know" if the card is doing legacy decoding like you.

The problem is that unless the card you are using is not doing any
legacy decoding (has been set not to decode legacy VGA IO ports and
legacy VGA memory), trying to use another card in the system that does
decode them requires switching off IO and MEM access on you. Even if you
aren't actually using those legacy stuff, the simple fact that the card
is decoding them mean it needs to be disabled.

(In fact, it could be avoided in some cases where the card is under a
different P2P bridge by just disabling VGA forwarding, but that's the
aribter salad, as a client, you don't have to "know" that).

So the arbiter provides calls to "lock" some resources and deals with
eventually ping-pong'ing IOs. It's currently providing APIs to lock
legacy IO and legacy MEM and to inform the arbiter that the card isn't
decoding some or all of the legacy stuff (to put it out of the arbiter
hands basically). It needs an addition to the API I posted for cases
where you don't know if the card is doing legacy decoding, to indicate
you want to access the normal memory/IO resources of the card, and
implictely lock the legacy ones if they are beeing decoded. I'm working
on it, I hope to have something to start playin with soon, there are
some issues to deal with the kernel vga console though that i'm trying
to iron out.

Ben.

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Mon, 2005-03-14 at 23:59 -0500, Michel Dänzer wrote:
> On Tue, 2005-03-15 at 09:37 +1100, Benjamin Herrenschmidt wrote:
> > > Be that as it may, it remains a fact that such a change would break
> > > existing installations...
> > > 
> > > I think that mode setting and memory allocation should be separated. X
> > > will always reserve enough video RAM for the largest resolution it uses
> > > for the screen contents.
> > 
> > But X has no control on where fbdev will allocate memory. 
> 
> My understanding is that so far, the fbdev API has pretty much implied
> that any mode scans out the beginning of the memory accessed via the
> framebuffer device, unless the panning ioctl is used. IIRC at least
> DirectFB makes basically the same assumptions as X there.

But that will not be true with dual head unless I put the second
framebuffer into some fixed location in memory, thus making life more
difficult for the allocator.

> I'm not suggesting that, but I do think that tying together mode
> switching and memory allocation would be a big mistake.
> 
> The EGL API for this is currently being discussed on the dri-egl list
> (http://lists.freedesktop.org/mailman/listinfo/dri-egl), you may want to
> join in there.

I'll try, I'm near by bandwidth limit already though.

> > We'll need to find a way to deal with that. An idea would be for me to 
> > do the clearing only when I come from a different bit depth or from text 
> > mode. That is, what i want to avoid, is those artifacts caused by 
> > incorrect bit depth content. A strict mode change isn't an issue in this 
> > case. That would solve my immediate problem.
> 
> Sounds good.
> 
> > _BUT_ in the long run, X will have to be smarter. Current UseFBDev or X
> > "fbdev" can't support dual head properly on top of fbdev anyway, 
> 
> Agreed for UseFBDev, but what's the problem with fbdev?

Read the rest of the email.
> 
> > But until all clients are DRI clients, we have a problem.
> 
> Keep in mind that basically the only driver-independent API exposed by
> the DRI is OpenGL, so I'm afraid it's unlikely that all fbdev
> applications will take that route.

I meant DRM. That is some arbitration & locking. There is simply _NO_
way we can have something that works with both independant multiple
heads and direct banging of the framebuffer without arbitration. There
is no magic here. It _can_ be made to work in _some_ cases using the
separate swappers, but that won't help if one of the heads try to do
accel....

Ben.

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Jon Smirl]

On Tue, 15 Mar 2005 09:37:53 +1100, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> 
> > Be that as it may, it remains a fact that such a change would break
> > existing installations...
> >
> > I think that mode setting and memory allocation should be separated. X
> > will always reserve enough video RAM for the largest resolution it uses
> > for the screen contents.
> 
> But X has no control on where fbdev will allocate memory. We are
> thinking with the "new model" in mind, and so far, a mode setting is
> under control of the framebuffer. Content of video memory (framebuffer,
> textures, overlay, whatever) simply cannot be considered as preserved
> accross mode switches.

I can agree that all video memory for that head can be booted. But
just because one head changes mode we shouldn't boot the objects for
the other head. You wouldn't want the two user case of one user
changing modes making the other user's display blink.

> 
> We can't also block all evolutions just because we have to support a
> broken model. We'll need to find a way to deal with that. An idea would
> be for me to do the clearing only when I come from a different bit depth
> or from text mode. That is, what i want to avoid, is those artifacts
> caused by incorrect bit depth content. A strict mode change isn't an
> issue in this case. That would solve my immediate problem.
> 
> _BUT_ in the long run, X will have to be smarter. Current UseFBDev or X
> "fbdev" can't support dual head properly on top of fbdev anyway, so that
> means I have some flexibility. The second head will probably" move on
> mode switches since I intend to allocate it at the top of the accessible
> aperture as described by Jon.

I agree that X has to be fixed to work cooperatively in this environment. 
The alternative is to just leave X alone and make all of this work for XGL.
The user would then choose which environment they want to run.

I'm leaning toward just leaving X alone and spending the resources on 
making XGL work. After all XGL is a complete X replacement.

If you want to run existing X load an old fbdev driver.  We can keep both
fbdev drivers in the kernel until it is clear if XGL is going mainstream.

One thing that does need to get fixed is mesa support for non-accelerated
fbdev drivers but mesa can add all of the appropriate locking.

> 
> All of that just keep uncovering more and more issues with our current
> fbdev model though. From discussions we had so far, it uncovers the
> problem of arbitration. That is, can we simply afford having a process
> mmap /dev/fb and blast things to it without any arbitration like we do
> today ?
> 
> If the answer is yes, then we are in deep trouble for lots of reason:
> 
>  - VGA Arbitration might require us to flip/flop MEM/IO enable bits
>  - Swapper control as explained earlier unless I can "reserve" a swapper
>    for each head, that is manage to have one aperture per head
>  - Engine discipline. What if the client on head 0 (like current X) uses
>    the engine ? Even if the one on head 1 doesn't, simple framebuffer
>    accesses can be enough to lock up the card.
>  - etc....
> 
> I think that Jon's dream of having totally independant heads that can
> run 2 separate applications is a long way away and we have sort-of to
> tie /dev/fb's together, though I don't know how to acheive that in
> practice, unless we switch to a new API that can enforce it. The current
> fbdev API cannot.

This is definitely something to works towards. There is a lot of
application for this in schools, libraries, internet cafe, etc. There
are several companies selling variations on this.

I don't expect an immediate solution but I want to make sure the
redesign allows it to be built.

> 
> DRI can do such things afaik (manage several contexts etc...), at least
> provides the infrastructure for it. But until all clients are DRI
> clients, we have a problem. That means that any direct fb client has to
> take over the entire card. All heads. There is simply no choice, and
> that doesn't even help with the VGA arbitration issue which still
> require explicit lock/unlock around accesses.
> 
> I'm open to suggestions...

Can we put in our own fault handler for the mmap, trap the directfb
accesses and do the proper locking?

> 
> Ben.
> 
> _______________________________________________
> xorg mailing list
> xorg@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/xorg
> 


-- 
Jon Smirl
jonsmirl@gmail.com

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Benjamin Herrenschmidt]

On Tue, 2005-03-15 at 00:30 -0500, Jon Smirl wrote:

> I agree that X has to be fixed to work cooperatively in this environment. 
> The alternative is to just leave X alone and make all of this work for XGL.
> The user would then choose which environment they want to run.
> 
> I'm leaning toward just leaving X alone and spending the resources on 
> making XGL work. After all XGL is a complete X replacement.
> 
> If you want to run existing X load an old fbdev driver.  We can keep both
> fbdev drivers in the kernel until it is clear if XGL is going mainstream.
> 
> One thing that does need to get fixed is mesa support for non-accelerated
> fbdev drivers but mesa can add all of the appropriate locking.

No, that isn't realistic. I want a smooth transition, and XGL won't be
the only player in the field anyway.
 
> > I think that Jon's dream of having totally independant heads that can
> > run 2 separate applications is a long way away and we have sort-of to
> > tie /dev/fb's together, though I don't know how to acheive that in
> > practice, unless we switch to a new API that can enforce it. The current
> > fbdev API cannot.
> 
> This is definitely something to works towards. There is a lot of
> application for this in schools, libraries, internet cafe, etc. There
> are several companies selling variations on this.
> 
> I don't expect an immediate solution but I want to make sure the
> redesign allows it to be built.

Yes. But I think that we should basically consider that anybody
openng /dev/fb also "locks out" write access to the other heads. That
is, the other head can be opened, but nothing done (mmap etc... on it)
unless both opener's have done the right ioctl telling us they know how
to do arbitration or that kind of thing. No ?

The fbdev API needs a mirror of the vga arbitration API. The later is
for use only by applications directly tapping vga legacy stuff. But
something using fbdev also need a lock/unlock API to enclose any
hardware access, at the very least. And when you think about it, what
you end up needing is ... the DRM lock.

So there might be some need to have fbdev be capable of taking the DRM
lock now. Unless that's done, I'll have to impose severe restrictions on
accesses to radeonfb's second head.

> Can we put in our own fault handler for the mmap, trap the directfb
> accesses and do the proper locking?

Gack. First that's "lazy locking" and something I'd like to avoid, but
if directfb API doesn't provide any other option ... then, it also
requires regulary un-mapping them back from the process using directfb
or the stuff will remain locked forever, and unmapping things behind a
process back isn't cool (some interesting locking issues). Overall
rather complicated and inefficient performance wise. And part of that
infrastructure already exist, somewhat, in the DRM ...

If a directFB client uses a /dev/fb, I think it should own both heads.
We simply can't do anything else. And we might even still need
arbitration because of the VGA stuff in case the card we are using has
legacy decoding enabled _anyway_. So ....


Ben.

Re: FB model basic issues (WAS: radeon, apertures & memory mapping)

[Ian Romanick]

Jon Smirl wrote:

> Can we put in our own fault handler for the mmap, trap the directfb
> accesses and do the proper locking?

Some SGI hardware used to work like that.  When they asked Linus for 
some kernel hooks to support that type of thing, well...I'm just glad 
*I* wasn't in the line of fire. ;)  I seriously doubt that it would be 
looked upon any kinder now.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: radeon, apertures & memory mapping

[Vladimir Dergachev]

Hi Ben,

   Hopefully simple question:

   If you were able to change configured aperture size - would you like to ? 
I could possibly imaging situation where this is used to regular access to 
the video memory, but you are the driver author..

                      thank you !

                            Vladimir Dergachev


On Sun, 13 Mar 2005, Benjamin Herrenschmidt wrote:

> Hi !
>
> I'm currently rewriting radeonfb to implement support for dual head, and
> ultimately, to make it more friendly to be hooked on DRM for mesa-solo
> style setups.
>
> I have some issues however related to the way memory is mapped and
> dealing with apertures. Here is the story, suggestions welcome:
>
> The radeon card exposes to the system 2 separate apertures. That is, the
> PCI region is actually cut by the hardware in two halves, each of them
> beeing an "aperture". Each aperture can have different configuration for
> the endian swappers (and possibly the surface tiling registers).
>
> I can configure the apertures to both map to the same bit of video
> memory (both covering the framebuffer from 0), or to be "split", that is
> aperture 0 covering the framebuffer from 0 to CONFIG_APER_SIZE (size of
> an aperture, that is half of the PCI BAR allocation), and aperture 1
> covering the framebuffer from CONFIG_APER_SIZE to CONFIG_APER_SIZE*2.
>
> However, I can't change anything to CONFIG_APER_SIZE itself, it's
> decided by straps, either HW or in the ROM. So we end up with different
> setups depending on how the BIOS has configured things. I know that
> Apple chips are usually wired so that CONFIG_APER_SIZE is half the video
> memory, so if I use the first mode, I can only access half of the video
> RAM from PCI, if I use the second, each aperture maps a different half
> of video memory with possibly different endian swapping.
>
> But I think the setups in real life are more diverse and some BIOSes
> will have CONFIG_APER_SIZE at least as big as the entire video memory,
> thus forcing me to use the "overlapped" setup. In fact, CONFIG_APER_SIZE
> may even be smaller than half of the vram and thus limiting the CPU to
> part of the VRAM anyway.
>
> I have toyed with all sort of setups, and I have +/- decided to not
> bother, and always do this, please tell me what you think:
>
> Always setup HOST_PATH_CNTL.HDP_APER_CNTL to 0. That is, both apertures
> are always overlapping. On Macs, or other machines that strap
> CONFIG_APER_SIZE to half of VRAM, that means only half of vram can be
> directly accessed by the CPU. I think this is fine because of these:
>
> - We only really need to bother about CPU access for the framebuffer
> itself (and possibly the cursor). That is normal non-accelerated fbdev
> operations an mmap'ing of the framebuffer in user space. This is not
> really a problem if that is limited to some part of vram. It puts a
> small constraint on the allocation of video memory: the framebuffer has
> to be near the beginning. But my opinion is that a mode switch will
> pretty much always invalidate everything that is cached in video memory,
> so the whole allocation can be re-done at that point. Things like
> texture uploads etc... should use the CP engine and DMA (from either
> system or AGP memory).
>
> - It's actually making things easier for me since I can allocate both
> framebuffers next to each other, and probably easier for the future
> video memory allocator since it then gets a larger contiguous chunk of
> memory to deal with.
>
> - It's also easier because If I wanted to take advantage of the "split"
> setting, I would still need some fallback mecanism for cards who have
> the aperture size set to all of vram size.
>
> Note that I also noticed rather inconsistent usage of memory size vs
> aperture size in DRI/X. I'm not sure X actually deals with the 2
> scenarios here (X always only uses aperture 0 for both heads and changes
> the swapper control on each access).
>
> The only drawback is that on the kernel side, for the console, I end up
> with the ioremap for the second crtc beeing twice as big as necessary
> since I can't "dynamically" ioremap/iounmap on mode switches (that would
> be guaranteed to fail after the system has been running for a while). It
> would be nice if I could play mapping tricks by doing something like
> allocating the 2 bits of virtual space at boot (get_vm_area), and just
> remap the pages in them ((un)map_vm_area), but that wouldn't work with
> architectures like MIPS who have limitations on where in virtual space
> non-cacheable things are.
>
> I could maybe use a single ioremap though, that is use a single
> aperture, and then switch the swapper on accesses. Though I should also
> be careful not to end up conflicting with a userland process relying on
> having the 2 separate aperture swappers stable for the mode on the 2
> separate framebuffer mappings... Like X would use fb0 while console
> would use fb1 with a different swapper setting. That would blow up for
> sure unless fbcon arbitrates accesses with X, which I don't see
> happening right away. I suppose we'll have to consider both heads linked
> as far as console ownership is concerned, at least for now, until the
> kernel console subsystem is overhauled significantly.
>
> Ben.
>
>
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> --
> _______________________________________________
> Dri-devel mailing list
> Dri-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dri-devel
>

Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

On Sat, 2005-03-12 at 22:55 -0500, Vladimir Dergachev wrote:
> Hi Ben,
> 
>    Hopefully simple question:
> 
>    If you were able to change configured aperture size - would you like to ? 
> I could possibly imaging situation where this is used to regular access to 
> the video memory, but you are the driver author..

You cannot change them. They are strapped on the mother board or in the
BIOS ROM. If I was able, I would probably set both apertures to cover
the entire VRAM, though ATI hints that you may have more flexibility
with tiled surfaces if using them split ...

>                       thank you !
> 
>                             Vladimir Dergachev
> 
> 
> On Sun, 13 Mar 2005, Benjamin Herrenschmidt wrote:
> 
> > Hi !
> >
> > I'm currently rewriting radeonfb to implement support for dual head, and
> > ultimately, to make it more friendly to be hooked on DRM for mesa-solo
> > style setups.
> >
> > I have some issues however related to the way memory is mapped and
> > dealing with apertures. Here is the story, suggestions welcome:
> >
> > The radeon card exposes to the system 2 separate apertures. That is, the
> > PCI region is actually cut by the hardware in two halves, each of them
> > beeing an "aperture". Each aperture can have different configuration for
> > the endian swappers (and possibly the surface tiling registers).
> >
> > I can configure the apertures to both map to the same bit of video
> > memory (both covering the framebuffer from 0), or to be "split", that is
> > aperture 0 covering the framebuffer from 0 to CONFIG_APER_SIZE (size of
> > an aperture, that is half of the PCI BAR allocation), and aperture 1
> > covering the framebuffer from CONFIG_APER_SIZE to CONFIG_APER_SIZE*2.
> >
> > However, I can't change anything to CONFIG_APER_SIZE itself, it's
> > decided by straps, either HW or in the ROM. So we end up with different
> > setups depending on how the BIOS has configured things. I know that
> > Apple chips are usually wired so that CONFIG_APER_SIZE is half the video
> > memory, so if I use the first mode, I can only access half of the video
> > RAM from PCI, if I use the second, each aperture maps a different half
> > of video memory with possibly different endian swapping.
> >
> > But I think the setups in real life are more diverse and some BIOSes
> > will have CONFIG_APER_SIZE at least as big as the entire video memory,
> > thus forcing me to use the "overlapped" setup. In fact, CONFIG_APER_SIZE
> > may even be smaller than half of the vram and thus limiting the CPU to
> > part of the VRAM anyway.
> >
> > I have toyed with all sort of setups, and I have +/- decided to not
> > bother, and always do this, please tell me what you think:
> >
> > Always setup HOST_PATH_CNTL.HDP_APER_CNTL to 0. That is, both apertures
> > are always overlapping. On Macs, or other machines that strap
> > CONFIG_APER_SIZE to half of VRAM, that means only half of vram can be
> > directly accessed by the CPU. I think this is fine because of these:
> >
> > - We only really need to bother about CPU access for the framebuffer
> > itself (and possibly the cursor). That is normal non-accelerated fbdev
> > operations an mmap'ing of the framebuffer in user space. This is not
> > really a problem if that is limited to some part of vram. It puts a
> > small constraint on the allocation of video memory: the framebuffer has
> > to be near the beginning. But my opinion is that a mode switch will
> > pretty much always invalidate everything that is cached in video memory,
> > so the whole allocation can be re-done at that point. Things like
> > texture uploads etc... should use the CP engine and DMA (from either
> > system or AGP memory).
> >
> > - It's actually making things easier for me since I can allocate both
> > framebuffers next to each other, and probably easier for the future
> > video memory allocator since it then gets a larger contiguous chunk of
> > memory to deal with.
> >
> > - It's also easier because If I wanted to take advantage of the "split"
> > setting, I would still need some fallback mecanism for cards who have
> > the aperture size set to all of vram size.
> >
> > Note that I also noticed rather inconsistent usage of memory size vs
> > aperture size in DRI/X. I'm not sure X actually deals with the 2
> > scenarios here (X always only uses aperture 0 for both heads and changes
> > the swapper control on each access).
> >
> > The only drawback is that on the kernel side, for the console, I end up
> > with the ioremap for the second crtc beeing twice as big as necessary
> > since I can't "dynamically" ioremap/iounmap on mode switches (that would
> > be guaranteed to fail after the system has been running for a while). It
> > would be nice if I could play mapping tricks by doing something like
> > allocating the 2 bits of virtual space at boot (get_vm_area), and just
> > remap the pages in them ((un)map_vm_area), but that wouldn't work with
> > architectures like MIPS who have limitations on where in virtual space
> > non-cacheable things are.
> >
> > I could maybe use a single ioremap though, that is use a single
> > aperture, and then switch the swapper on accesses. Though I should also
> > be careful not to end up conflicting with a userland process relying on
> > having the 2 separate aperture swappers stable for the mode on the 2
> > separate framebuffer mappings... Like X would use fb0 while console
> > would use fb1 with a different swapper setting. That would blow up for
> > sure unless fbcon arbitrates accesses with X, which I don't see
> > happening right away. I suppose we'll have to consider both heads linked
> > as far as console ownership is concerned, at least for now, until the
> > kernel console subsystem is overhauled significantly.
> >
> > Ben.
> >
> >
> >
> >
> > -------------------------------------------------------
> > SF email is sponsored by - The IT Product Guide
> > Read honest & candid reviews on hundreds of IT Products from real users.
> > Discover which products truly live up to the hype. Start reading now.
> > http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> > --
> > _______________________________________________
> > Dri-devel mailing list
> > Dri-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/dri-devel
> >
-- 
Benjamin Herrenschmidt <benh@kernel.crashing.org>

Re: radeon, apertures & memory mapping

[Ville Syrjälä]

On Sun, Mar 13, 2005 at 12:35:43PM +1100, Benjamin Herrenschmidt wrote:
> Hi !
> 
> I'm currently rewriting radeonfb to implement support for dual head, and
> ultimately, to make it more friendly to be hooked on DRM for mesa-solo
> style setups.
> 
> I have some issues however related to the way memory is mapped and
> dealing with apertures. Here is the story, suggestions welcome:
> 
> The radeon card exposes to the system 2 separate apertures. That is, the
> PCI region is actually cut by the hardware in two halves, each of them
> beeing an "aperture". Each aperture can have different configuration for
> the endian swappers (and possibly the surface tiling registers).
> 
> I can configure the apertures to both map to the same bit of video
> memory (both covering the framebuffer from 0), or to be "split", that is
> aperture 0 covering the framebuffer from 0 to CONFIG_APER_SIZE (size of
> an aperture, that is half of the PCI BAR allocation), and aperture 1
> covering the framebuffer from CONFIG_APER_SIZE to CONFIG_APER_SIZE*2.
> 
> However, I can't change anything to CONFIG_APER_SIZE itself, it's
> decided by straps, either HW or in the ROM. So we end up with different
> setups depending on how the BIOS has configured things. I know that
> Apple chips are usually wired so that CONFIG_APER_SIZE is half the video
> memory, so if I use the first mode, I can only access half of the video
> RAM from PCI, if I use the second, each aperture maps a different half
> of video memory with possibly different endian swapping.
> 
> But I think the setups in real life are more diverse and some BIOSes
> will have CONFIG_APER_SIZE at least as big as the entire video memory,
> thus forcing me to use the "overlapped" setup. In fact, CONFIG_APER_SIZE
> may even be smaller than half of the vram and thus limiting the CPU to
> part of the VRAM anyway.
> 
> I have toyed with all sort of setups, and I have +/- decided to not
> bother, and always do this, please tell me what you think:
> 
> Always setup HOST_PATH_CNTL.HDP_APER_CNTL to 0. That is, both apertures
> are always overlapping. On Macs, or other machines that strap
> CONFIG_APER_SIZE to half of VRAM, that means only half of vram can be
> directly accessed by the CPU. I think this is fine because of these:
> 
>  - We only really need to bother about CPU access for the framebuffer
> itself (and possibly the cursor). That is normal non-accelerated fbdev
> operations an mmap'ing of the framebuffer in user space. This is not
> really a problem if that is limited to some part of vram. It puts a
> small constraint on the allocation of video memory: the framebuffer has
> to be near the beginning.

It will limit DirectFB to access only CONFIG_APER_SIZE. DirectFB needs CPU 
access to offscreen memory for software fallbacks and explicit user 
access. Any other compositing window system would need to do the same. If 
the video memory manager ever gets done then it shouldn't be a major 
problem because the kernel could blit the data to/from the inaccesible 
part without the application even realizing it. Although direct access 
might be useful in that case also since it could reduce pressure on the 
GART address space.

> But my opinion is that a mode switch will
> pretty much always invalidate everything that is cached in video memory,

I don't see any reason for such a sledgehammer approach. If the new mode 
doesn't overlap with any offscreen data then there is no need to 
invalidate anything.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click

Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

On Sun, 2005-03-13 at 10:22 +0200, Ville Syrjälä wrote:

> > 
> >  - We only really need to bother about CPU access for the framebuffer
> > itself (and possibly the cursor). That is normal non-accelerated fbdev
> > operations an mmap'ing of the framebuffer in user space. This is not
> > really a problem if that is limited to some part of vram. It puts a
> > small constraint on the allocation of video memory: the framebuffer has
> > to be near the beginning.
> 
> It will limit DirectFB to access only CONFIG_APER_SIZE. DirectFB needs CPU 
> access to offscreen memory for software fallbacks and explicit user 
> access. Any other compositing window system would need to do the same. If 
> the video memory manager ever gets done then it shouldn't be a major 
> problem because the kernel could blit the data to/from the inaccesible 
> part without the application even realizing it. Although direct access 
> might be useful in that case also since it could reduce pressure on the 
> GART address space.

Yes, that means direct access will be limited to half of the vram on
some setups and not on others, depending on how the BIOS sets up the
card. Is this a real issue ? I don't think so personally. Especially
since directfb could make use of DRM to do DMA blits either from main
memory or from AGP space... Or things can be put in accessible space,
and blitted elsewhere using the accelerator.

That "half" of vram is plenty enough for a framebuffer (and more). it's
only an issue when you start doing very large offscreen surfaces. Do you
have much usage of those without DMA ?
 
> > But my opinion is that a mode switch will
> > pretty much always invalidate everything that is cached in video memory,
> 
> I don't see any reason for such a sledgehammer approach. If the new mode 
> doesn't overlap with any offscreen data then there is no need to 
> invalidate anything.

Depends how smart the memory manager is.

Ben.

Re: radeon, apertures & memory mapping

[Ville Syrjälä]

On Sun, Mar 13, 2005 at 08:20:45PM +1100, Benjamin Herrenschmidt wrote:
> On Sun, 2005-03-13 at 10:22 +0200, Ville Syrjälä wrote:
> 
> > > 
> > >  - We only really need to bother about CPU access for the framebuffer
> > > itself (and possibly the cursor). That is normal non-accelerated fbdev
> > > operations an mmap'ing of the framebuffer in user space. This is not
> > > really a problem if that is limited to some part of vram. It puts a
> > > small constraint on the allocation of video memory: the framebuffer has
> > > to be near the beginning.
> > 
> > It will limit DirectFB to access only CONFIG_APER_SIZE. DirectFB needs CPU 
> > access to offscreen memory for software fallbacks and explicit user 
> > access. Any other compositing window system would need to do the same. If 
> > the video memory manager ever gets done then it shouldn't be a major 
> > problem because the kernel could blit the data to/from the inaccesible 
> > part without the application even realizing it. Although direct access 
> > might be useful in that case also since it could reduce pressure on the 
> > GART address space.
> 
> Yes, that means direct access will be limited to half of the vram on
> some setups and not on others, depending on how the BIOS sets up the
> card. Is this a real issue ? I don't think so personally. Especially
> since directfb could make use of DRM to do DMA blits either from main
> memory or from AGP space...

AGP as it's currently used is pretty much pointless for software fallbacks 
since reading from AGP memory is nearly as slow as reading from video 
memory.

> Or things can be put in accessible space,
> and blitted elsewhere using the accelerator.

This could work (and it would avoid DRM which in my book is a plus) but 
it's not very nice to have to copy the data twice.

> That "half" of vram is plenty enough for a framebuffer (and more). it's
> only an issue when you start doing very large offscreen surfaces. Do you
> have much usage of those without DMA ?

I have about 26MB of video memory used when running XDirectFB with GNOME, 
epiphany and 4 gnome-terminals, and I also have some videos playing on the 
TV at the same time. That's on a 32MB G400 BTW.


I must be missing something something obvious because I don't quite 
understand what major drawbacks there are with the non-overlapping mode. 
As I see it you get at least the same amount of CPU accessible memory as 
you get in the overlapping mode.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click
--

Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

> AGP as it's currently used is pretty much pointless for software fallbacks 
> since reading from AGP memory is nearly as slow as reading from video 
> memory.

Hrm.. I wouldn't expect _that_ slow. It's uncacheable, right, but still
on a faster bus. Especially if we use it the way we do on ppc where we
actually map the RAM pages directly instead of having processes go
through the GART.

> > Or things can be put in accessible space,
> > and blitted elsewhere using the accelerator.
> 
> This could work (and it would avoid DRM which in my book is a plus) but 
> it's not very nice to have to copy the data twice.
> 
> > That "half" of vram is plenty enough for a framebuffer (and more). it's
> > only an issue when you start doing very large offscreen surfaces. Do you
> > have much usage of those without DMA ?
> 
> I have about 26MB of video memory used when running XDirectFB with GNOME, 
> epiphany and 4 gnome-terminals, and I also have some videos playing on the 
> TV at the same time. That's on a 32MB G400 BTW.
> 
> I must be missing something something obvious because I don't quite 
> understand what major drawbacks there are with the non-overlapping mode. 
> As I see it you get at least the same amount of CPU accessible memory as 
> you get in the overlapping mode.

Yes, you do, but that means that if the apertures are configured such
that the entire VRAM fits in a single aperture, then you just can't use
the second aperture at all. Which means you can't have separate swapper
setting for both apertures, and thus, can't let two independant
processes access the video memory with different bit depth, at least on
big endian machines unless you do trickery, and play with the swapper
before each access.

Ben.

Re: radeon, apertures & memory mapping

[Jon Smirl]

On Sun, 13 Mar 2005 23:04:59 +1100, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> 
> > AGP as it's currently used is pretty much pointless for software fallbacks
> > since reading from AGP memory is nearly as slow as reading from video
> > memory.
> 
> Hrm.. I wouldn't expect _that_ slow. It's uncacheable, right, but still
> on a faster bus. Especially if we use it the way we do on ppc where we
> actually map the RAM pages directly instead of having processes go
> through the GART.

I asked at the Xdev conference if there were page table tricks that
would work for accessing GART memory.  Everybody said no but I'm still
wondering if there are any.

For example the ppc has an instruction for flushing specific pages
from cache, unlike the x86 where you can only flush everything.

So on the ppc you could leave the GART memory mapped normally and
cached. Do all of your fallback calculations, then flush the address
range from cache. Now tell the GPU to go use it.

Can't GART memory be normally cached RAM as long as we flush the cache
before telling the GPU to use it?

If you are doing fallback calculations in a 6MB buffer that is 1,500
pages. Accessing all of this effectively flushes the data cache. Once
you are done with it you probably don't want those pages in the cache
anyway.

-- 
Jon Smirl
jonsmirl@gmail.com

Re: Re: radeon, apertures & memory mapping

[Ville Syrjälä]

On Sun, Mar 13, 2005 at 11:19:35AM -0500, Jon Smirl wrote:
> On Sun, 13 Mar 2005 23:04:59 +1100, Benjamin Herrenschmidt
> <benh@kernel.crashing.org> wrote:
> > 
> > > AGP as it's currently used is pretty much pointless for software fallbacks
> > > since reading from AGP memory is nearly as slow as reading from video
> > > memory.
> > 
> > Hrm.. I wouldn't expect _that_ slow. It's uncacheable, right, but still
> > on a faster bus. Especially if we use it the way we do on ppc where we
> > actually map the RAM pages directly instead of having processes go
> > through the GART.
> 
> I asked at the Xdev conference if there were page table tricks that
> would work for accessing GART memory.  Everybody said no but I'm still
> wondering if there are any.
> 
> For example the ppc has an instruction for flushing specific pages
> from cache, unlike the x86 where you can only flush everything.
> 
> So on the ppc you could leave the GART memory mapped normally and
> cached. Do all of your fallback calculations, then flush the address
> range from cache. Now tell the GPU to go use it.
> 
> Can't GART memory be normally cached RAM as long as we flush the cache
> before telling the GPU to use it?
> 
> If you are doing fallback calculations in a 6MB buffer that is 1,500
> pages. Accessing all of this effectively flushes the data cache. Once
> you are done with it you probably don't want those pages in the cache
> anyway.

I don't understand why we have "GART memory" anyway. It's just main memory 
and I don't see any point going through the GART to access it with the 
CPU. Only the graphics card needs to use the GART.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Jon Smirl]

On Sun, 13 Mar 2005 19:47:14 +0200, Ville Syrjälä <syrjala@sci.fi> wrote:
> I don't understand why we have "GART memory" anyway. It's just main memory
> and I don't see any point going through the GART to access it with the
> CPU. Only the graphics card needs to use the GART.

I see no need to for the CPU to go through the GART either. The main
CPU page tables can provide the same rearranging that the GART does.

We do need specially marked GART memory because of caching issues. If
the CPU writes to GART RAM the write may still be on the CPU chip in a
cache. We have to make sure it gets pushed into physical memory so
that the GPU can see it.

The best model would be to chuck the AGP/PCI Express interface on the
board and have a hyperchannel instead. Hyperchannel provides full
cache consistency without all of these flushing problems. The GPU
really is another specialized CPU, give it a CPU class memory
interface.

-- 
Jon Smirl
jonsmirl@gmail.com


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click
--

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

> The best model would be to chuck the AGP/PCI Express interface on the
> board and have a hyperchannel instead. Hyperchannel provides full
> cache consistency without all of these flushing problems. The GPU
> really is another specialized CPU, give it a CPU class memory
> interface.

You mean HyperTransport ? Well, PCI Express isn't far from that
neither...

Ben.

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Jon Smirl]

On Mon, 14 Mar 2005 08:52:46 +1100, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> 
> > The best model would be to chuck the AGP/PCI Express interface on the
> > board and have a hyperchannel instead. Hyperchannel provides full
> > cache consistency without all of these flushing problems. The GPU
> > really is another specialized CPU, give it a CPU class memory
> > interface.
> 
> You mean HyperTransport ? Well, PCI Express isn't far from that
> neither...

Yes, HyperTransport. That's what I get for writing email while the
babies are crying.

Does PCI Express support a full cache coherency protocol like
HyperTransport? So if the GPU touches a page of system memory, the
system memory will get flushed out of the CPU cache? In that case we
don't need to mark shared GPU/CPU memory as non-cachable.

> 
> Ben.
> 
> 


-- 
Jon Smirl
jonsmirl@gmail.com


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
--

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

On Sun, 2005-03-13 at 17:17 -0500, Jon Smirl wrote:
> On Mon, 14 Mar 2005 08:52:46 +1100, Benjamin Herrenschmidt
> <benh@kernel.crashing.org> wrote:
> > 
> > > The best model would be to chuck the AGP/PCI Express interface on the
> > > board and have a hyperchannel instead. Hyperchannel provides full
> > > cache consistency without all of these flushing problems. The GPU
> > > really is another specialized CPU, give it a CPU class memory
> > > interface.
> > 
> > You mean HyperTransport ? Well, PCI Express isn't far from that
> > neither...
> 
> Yes, HyperTransport. That's what I get for writing email while the
> babies are crying.
> 
> Does PCI Express support a full cache coherency protocol like
> HyperTransport?

Or just like PCI :)

AGP is just a special case, and I'm not even sure wether the cache
incoherency is a spec thing or just that all implementations are
broken :) I'll check when I find my AGP spec.

>  So if the GPU touches a page of system memory, the
> system memory will get flushed out of the CPU cache? In that case we
> don't need to mark shared GPU/CPU memory as non-cachable.

Well, that's what happen with a normal cache coherent setup, but I'm
fairly sure that won't happen with AGP on a lot of machines.

Ben.




-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
--

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

> > If you are doing fallback calculations in a 6MB buffer that is 1,500
> > pages. Accessing all of this effectively flushes the data cache. Once
> > you are done with it you probably don't want those pages in the cache
> > anyway.
> 
> I don't understand why we have "GART memory" anyway. It's just main memory 
> and I don't see any point going through the GART to access it with the 
> CPU. Only the graphics card needs to use the GART.

Which is what we do on some archs, like Apple ppcs, where the GART
doesn't work for CPU accesses. We just use the real RAM pages. However,
we have to map them non-cacheable since the GPU GART accesses bypass the
cache coherency protocol (that is the case on most AGP implementations
afaik and is broken by design, imho).

Ben.

Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

On Sun, 2005-03-13 at 11:19 -0500, Jon Smirl wrote:
> On Sun, 13 Mar 2005 23:04:59 +1100, Benjamin Herrenschmidt
> <benh@kernel.crashing.org> wrote:
> > 
> > > AGP as it's currently used is pretty much pointless for software fallbacks
> > > since reading from AGP memory is nearly as slow as reading from video
> > > memory.
> > 
> > Hrm.. I wouldn't expect _that_ slow. It's uncacheable, right, but still
> > on a faster bus. Especially if we use it the way we do on ppc where we
> > actually map the RAM pages directly instead of having processes go
> > through the GART.
> 
> I asked at the Xdev conference if there were page table tricks that
> would work for accessing GART memory.  Everybody said no but I'm still
> wondering if there are any.
> 
> For example the ppc has an instruction for flushing specific pages
> from cache, unlike the x86 where you can only flush everything.
> 
> So on the ppc you could leave the GART memory mapped normally and
> cached. Do all of your fallback calculations, then flush the address
> range from cache. Now tell the GPU to go use it.
> 
> Can't GART memory be normally cached RAM as long as we flush the cache
> before telling the GPU to use it?

For CPU -> GPU transfers, yes, but I'm not sure it would be that
efficient to do so. We have instructions to flush a cache line. A whole
page would require a bunch of them, not sure i would be that efficient
but worth benching maybe.
> 
> If you are doing fallback calculations in a 6MB buffer that is 1,500
> pages. Accessing all of this effectively flushes the data cache. Once
> you are done with it you probably don't want those pages in the cache
> anyway.

I wouldn't count on it flushing anything

Ben.

Re: radeon, apertures & memory mapping

[Jon Smirl]

On Mon, 14 Mar 2005 08:49:13 +1100, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> > If you are doing fallback calculations in a 6MB buffer that is 1,500
> > pages. Accessing all of this effectively flushes the data cache. Once
> > you are done with it you probably don't want those pages in the cache
> > anyway.
> 
> I wouldn't count on it flushing anything

I meant flushes out everything except the 1,500 pages you just
accessed. Since you don't want those pages any more a total cache
flush shouldn't make a difference, you don't want any of these pages
in the cache anyway.

> 
> Ben.
> 
> 


-- 
Jon Smirl
jonsmirl@gmail.com


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
--

Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

On Sun, 2005-03-13 at 17:10 -0500, Jon Smirl wrote:
> On Mon, 14 Mar 2005 08:49:13 +1100, Benjamin Herrenschmidt
> <benh@kernel.crashing.org> wrote:
> > > If you are doing fallback calculations in a 6MB buffer that is 1,500
> > > pages. Accessing all of this effectively flushes the data cache. Once
> > > you are done with it you probably don't want those pages in the cache
> > > anyway.
> > 
> > I wouldn't count on it flushing anything
> 
> I meant flushes out everything except the 1,500 pages you just
> accessed. Since you don't want those pages any more a total cache
> flush shouldn't make a difference, you don't want any of these pages
> in the cache anyway.

I wouldn't count on it again. Not all caches have a strict PLRU
algorithm, some caches do random replacement (or a mix of those), some
CPUs do agressive speculative loads and may bring back stuffs in the
cache just for fun, etc ....

Though the flushes may be fast if there is no actual hit in the cache, I
agree. Again, that should be benched.

In fact, i would _love_ to be able to mark AGP memory as cacheable on
ppc, even if there is no performance benefit in the end. The issue is
that currently, we end up having both a cacheable and a non-cacheable
mapping for those pages (the kernel linear mapping still maps those
pages cacheable, and it's almost impossible to get rid of that unless
you are prepared to disable the large pages mapping of kernel space or
the BATs on ppc32, which would harm kernel performances significantly).

It works, but it's illegal. That means that the CPU might well speculate
a load from one of these pages in kernel-land just because it happens to
be next to a page where you are iterating an array, and may then bring a
bit in the cache from that page.

At that point, a non-cacheable access from userland to that same line
that was brought to the cache may lead to undefined behaviour, ranging
from just works, to checkstops the CPU, with cases of writing corrupted
data, etc... depending on the CPU.

I yet have to see the problem happening in practice, but we are
definitely not on the safe side currently. I suspect ppc32 in practice
won't hit it, but ppc64 will...

Ben.

Re: radeon, apertures & memory mapping

[Jon Smirl]

On Mon, 14 Mar 2005 09:20:01 +1100, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> On Sun, 2005-03-13 at 17:10 -0500, Jon Smirl wrote:
> > On Mon, 14 Mar 2005 08:49:13 +1100, Benjamin Herrenschmidt
> > <benh@kernel.crashing.org> wrote:
> > > > If you are doing fallback calculations in a 6MB buffer that is 1,500
> > > > pages. Accessing all of this effectively flushes the data cache. Once
> > > > you are done with it you probably don't want those pages in the cache
> > > > anyway.
> > >
> > > I wouldn't count on it flushing anything
> >
> > I meant flushes out everything except the 1,500 pages you just
> > accessed. Since you don't want those pages any more a total cache
> > flush shouldn't make a difference, you don't want any of these pages
> > in the cache anyway.
> 
> I wouldn't count on it again. Not all caches have a strict PLRU
> algorithm, some caches do random replacement (or a mix of those), some
> CPUs do agressive speculative loads and may bring back stuffs in the
> cache just for fun, etc ....

I'm not being clear....

Leave AGP memory as normal RAM
driver does it thing to the memory
driver executes flush of data cache on CPU
after flush tell GPU to access the data

The performance hit of executing the flush is probably negligible
since you probably didn't care about anything in the data cache. All
of those entries would be replaced by later code anyway. You will lose
some later overlap parallelism as the cache is refilled.

> 
> Though the flushes may be fast if there is no actual hit in the cache, I
> agree. Again, that should be benched.
> 
> In fact, i would _love_ to be able to mark AGP memory as cacheable on
> ppc, even if there is no performance benefit in the end. The issue is
> that currently, we end up having both a cacheable and a non-cacheable
> mapping for those pages (the kernel linear mapping still maps those
> pages cacheable, and it's almost impossible to get rid of that unless
> you are prepared to disable the large pages mapping of kernel space or
> the BATs on ppc32, which would harm kernel performances significantly).
> 
> It works, but it's illegal. That means that the CPU might well speculate
> a load from one of these pages in kernel-land just because it happens to
> be next to a page where you are iterating an array, and may then bring a
> bit in the cache from that page.

That shouldn't matter the page brought in would be for a speculative
read and never accessed. It should just fall out of the cache and not
be written back. There is only one cachable mapping. In this model
writes are always followed by a flush before telling the GPU to access
the memory that has just been written.

> 
> At that point, a non-cacheable access from userland to that same line
> that was brought to the cache may lead to undefined behaviour, ranging
> from just works, to checkstops the CPU, with cases of writing corrupted
> data, etc... depending on the CPU.
> 
> I yet have to see the problem happening in practice, but we are
> definitely not on the safe side currently. I suspect ppc32 in practice
> won't hit it, but ppc64 will...
> 
> Ben.
> 
> 


-- 
Jon Smirl
jonsmirl@gmail.com

Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

> I'm not being clear....
> 
> Leave AGP memory as normal RAM
> driver does it thing to the memory
> driver executes flush of data cache on CPU
> after flush tell GPU to access the data
> 
> The performance hit of executing the flush is probably negligible
> since you probably didn't care about anything in the data cache. All
> of those entries would be replaced by later code anyway. You will lose
> some later overlap parallelism as the cache is refilled.

Should be measured though, but yes. I agree. We must make sure we have a
proper hook in the userland DRI to flush AGP pages before they get
submited (indirect buffers, texture datas, host data blits, ...) and the
kernel DRM should flush ring entries (easy probably to do it from the
various ring access macros).
> > 
> > Though the flushes may be fast if there is no actual hit in the cache, I
> > agree. Again, that should be benched.
> > 
> > In fact, i would _love_ to be able to mark AGP memory as cacheable on
> > ppc, even if there is no performance benefit in the end. The issue is
> > that currently, we end up having both a cacheable and a non-cacheable
> > mapping for those pages (the kernel linear mapping still maps those
> > pages cacheable, and it's almost impossible to get rid of that unless
> > you are prepared to disable the large pages mapping of kernel space or
> > the BATs on ppc32, which would harm kernel performances significantly).
> > 
> > It works, but it's illegal. That means that the CPU might well speculate
> > a load from one of these pages in kernel-land just because it happens to
> > be next to a page where you are iterating an array, and may then bring a
> > bit in the cache from that page.
> 
> That shouldn't matter the page brought in would be for a speculative
> read and never accessed. It should just fall out of the cache and not
> be written back. There is only one cachable mapping. In this model
> writes are always followed by a flush before telling the GPU to access
> the memory that has just been written.

I was talking about the current state of having both cacheable and
non-cacheable mappng. I was saying that this model has the above
possible issue, and that indeed, mapping everything cacheable would fix
it.
> > 
> > At that point, a non-cacheable access from userland to that same line
> > that was brought to the cache may lead to undefined behaviour, ranging
> > from just works, to checkstops the CPU, with cases of writing corrupted
> > data, etc... depending on the CPU.
> > 
> > I yet have to see the problem happening in practice, but we are
> > definitely not on the safe side currently. I suspect ppc32 in practice
> > won't hit it, but ppc64 will...
> > 
> > Ben.
> > 
> > 
> 
> 
-- 
Benjamin Herrenschmidt <benh@kernel.crashing.org>

Re: radeon, apertures & memory mapping

[Ville Syrjälä]

On Sun, Mar 13, 2005 at 06:00:01PM -0500, Jon Smirl wrote:
> On Mon, 14 Mar 2005 09:20:01 +1100, Benjamin Herrenschmidt
> <benh@kernel.crashing.org> wrote:
> > Though the flushes may be fast if there is no actual hit in the cache, I
> > agree. Again, that should be benched.
> > 
> > In fact, i would _love_ to be able to mark AGP memory as cacheable on
> > ppc, even if there is no performance benefit in the end. The issue is
> > that currently, we end up having both a cacheable and a non-cacheable
> > mapping for those pages (the kernel linear mapping still maps those
> > pages cacheable, and it's almost impossible to get rid of that unless
> > you are prepared to disable the large pages mapping of kernel space or
> > the BATs on ppc32, which would harm kernel performances significantly).
> > 
> > It works, but it's illegal. That means that the CPU might well speculate
> > a load from one of these pages in kernel-land just because it happens to
> > be next to a page where you are iterating an array, and may then bring a
> > bit in the cache from that page.
> 
> That shouldn't matter the page brought in would be for a speculative
> read and never accessed. It should just fall out of the cache and not
> be written back. There is only one cachable mapping. In this model
> writes are always followed by a flush before telling the GPU to access
> the memory that has just been written.

What about this scenario?

Speculative read -> AGP master writes new data -> CPU has invalid data in 
cache :(

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

> > That shouldn't matter the page brought in would be for a speculative
> > read and never accessed. It should just fall out of the cache and not
> > be written back. There is only one cachable mapping. In this model
> > writes are always followed by a flush before telling the GPU to access
> > the memory that has just been written.
> 
> What about this scenario?
> 
> Speculative read -> AGP master writes new data -> CPU has invalid data in 
> cache :(

First, we must be very careful with AGP master writes. I don't know if
we do a lot of them currently, but I know a collection of north bridges
that do not support them.

(Which is interesting, that means that if we want to copy something out
of video memory, we can't write it to AGP memory and then read it, we
need to actually do the blit from the CPU, good to know for our memory
manager. That also means that we have a problem if the video memory
isn't entirely accessible by the CPU ...)

That's something we should probably think about doing properly: Have a
list of AGP "issues" (errata ?) bits that are communicated by the AGP
host driver to the DRM.

At least all the early Apple AGP bridges don't do writes, and I remember
we have trouble with a few x86 ones as well. There are also issues when
a single AGP burst crosses a page boundary, and other things like that.

Ben.

Re: radeon, apertures & memory mapping

[Ville Syrjälä]

On Mon, Mar 14, 2005 at 10:48:19AM +1100, Benjamin Herrenschmidt wrote:
> 
> > > That shouldn't matter the page brought in would be for a speculative
> > > read and never accessed. It should just fall out of the cache and not
> > > be written back. There is only one cachable mapping. In this model
> > > writes are always followed by a flush before telling the GPU to access
> > > the memory that has just been written.
> > 
> > What about this scenario?
> > 
> > Speculative read -> AGP master writes new data -> CPU has invalid data in 
> > cache :(
> 
> First, we must be very careful with AGP master writes. I don't know if
> we do a lot of them currently, but I know a collection of north bridges
> that do not support them.

I don't think "normal" drivers do them at all. I did experiment with 
DirectFB at one point and had it place all offscreen surfaces to AGP 
memory. It worked really well on my hardware (G400 + VIA KT133 
northbridge). I also tried it with PCI transfers and that too worked but 
was naturally slower. I'd like to make DirectFB use AGP again since 32MB 
of video memory isn't always enough.

> (Which is interesting, that means that if we want to copy something out
> of video memory, we can't write it to AGP memory and then read it, we
> need to actually do the blit from the CPU, good to know for our memory
> manager. That also means that we have a problem if the video memory
> isn't entirely accessible by the CPU ...)

What about PCI master writes? Are there bridges that don't support even 
those?

> That's something we should probably think about doing properly: Have a
> list of AGP "issues" (errata ?) bits that are communicated by the AGP
> host driver to the DRM.
> 
> At least all the early Apple AGP bridges don't do writes, and I remember
> we have trouble with a few x86 ones as well. There are also issues when
> a single AGP burst crosses a page boundary, and other things like that.

:(

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click
--

Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

> I don't think "normal" drivers do them at all. I did experiment with 
> DirectFB at one point and had it place all offscreen surfaces to AGP 
> memory. It worked really well on my hardware (G400 + VIA KT133 
> northbridge). I also tried it with PCI transfers and that too worked but 
> was naturally slower. I'd like to make DirectFB use AGP again since 32MB 
> of video memory isn't always enough.
> 
> > (Which is interesting, that means that if we want to copy something out
> > of video memory, we can't write it to AGP memory and then read it, we
> > need to actually do the blit from the CPU, good to know for our memory
> > manager. That also means that we have a problem if the video memory
> > isn't entirely accessible by the CPU ...)
> 
> What about PCI master writes? Are there bridges that don't support even 
> those?

Usually, those work. I have a list of errata of Apple bridges at least
in Darwin, some do not support PCI master writes, but it's only
apparently some never-released revisions :)

> > That's something we should probably think about doing properly: Have a
> > list of AGP "issues" (errata ?) bits that are communicated by the AGP
> > host driver to the DRM.
> > 
> > At least all the early Apple AGP bridges don't do writes, and I remember
> > we have trouble with a few x86 ones as well. There are also issues when
> > a single AGP burst crosses a page boundary, and other things like that.
> 
> :(
> 
-- 
Benjamin Herrenschmidt <benh@kernel.crashing.org>

Re: radeon, apertures & memory mapping

[Jon Smirl]

On Mon, 14 Mar 2005 10:48:19 +1100, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> 
> > > That shouldn't matter the page brought in would be for a speculative
> > > read and never accessed. It should just fall out of the cache and not
> > > be written back. There is only one cachable mapping. In this model
> > > writes are always followed by a flush before telling the GPU to access
> > > the memory that has just been written.
> >
> > What about this scenario?
> >
> > Speculative read -> AGP master writes new data -> CPU has invalid data in
> > cache :(
> 

You need to reverse the cache flush process if you are going to read
data written by the GPU.

1) Make sure GPU is finished writing
2) flush your cache
3) read AGP memory like normal RAM.

-- 
Jon Smirl
jonsmirl@gmail.com


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
--

Re: radeon, apertures & memory mapping

[Ville Syrjälä]

On Sun, Mar 13, 2005 at 07:25:15PM -0500, Jon Smirl wrote:
> On Mon, 14 Mar 2005 10:48:19 +1100, Benjamin Herrenschmidt
> <benh@kernel.crashing.org> wrote:
> > 
> > > > That shouldn't matter the page brought in would be for a speculative
> > > > read and never accessed. It should just fall out of the cache and not
> > > > be written back. There is only one cachable mapping. In this model
> > > > writes are always followed by a flush before telling the GPU to access
> > > > the memory that has just been written.
> > >
> > > What about this scenario?
> > >
> > > Speculative read -> AGP master writes new data -> CPU has invalid data in
> > > cache :(
> > 
> 
> You need to reverse the cache flush process if you are going to read
> data written by the GPU.
> 
> 1) Make sure GPU is finished writing
> 2) flush your cache
> 3) read AGP memory like normal RAM.

Oh right. The CPU shouldn't write back the cached data since it hasn't 
changed.

I think you'd also need the GPU to issue an AGP flush command between 
steps 1 and 2.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click

Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

On Mon, 2005-03-14 at 02:39 +0200, Ville Syrjälä wrote:
> On Sun, Mar 13, 2005 at 07:25:15PM -0500, Jon Smirl wrote:
> > On Mon, 14 Mar 2005 10:48:19 +1100, Benjamin Herrenschmidt
> > <benh@kernel.crashing.org> wrote:
> > > 
> > > > > That shouldn't matter the page brought in would be for a speculative
> > > > > read and never accessed. It should just fall out of the cache and not
> > > > > be written back. There is only one cachable mapping. In this model
> > > > > writes are always followed by a flush before telling the GPU to access
> > > > > the memory that has just been written.
> > > >
> > > > What about this scenario?
> > > >
> > > > Speculative read -> AGP master writes new data -> CPU has invalid data in
> > > > cache :(
> > > 
> > 
> > You need to reverse the cache flush process if you are going to read
> > data written by the GPU.
> > 
> > 1) Make sure GPU is finished writing
> > 2) flush your cache
> > 3) read AGP memory like normal RAM.
> 
> Oh right. The CPU shouldn't write back the cached data since it hasn't 
> changed.
> 
> I think you'd also need the GPU to issue an AGP flush command between 
> steps 1 and 2.

Not exactly, it could wait before 3) in fact. Step 2 will just
"invalidate" the cache.

Ben.

Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

On Sun, 2005-03-13 at 19:25 -0500, Jon Smirl wrote:
> On Mon, 14 Mar 2005 10:48:19 +1100, Benjamin Herrenschmidt
> <benh@kernel.crashing.org> wrote:
> > 
> > > > That shouldn't matter the page brought in would be for a speculative
> > > > read and never accessed. It should just fall out of the cache and not
> > > > be written back. There is only one cachable mapping. In this model
> > > > writes are always followed by a flush before telling the GPU to access
> > > > the memory that has just been written.
> > >
> > > What about this scenario?
> > >
> > > Speculative read -> AGP master writes new data -> CPU has invalid data in
> > > cache :(
> > 
> 
> You need to reverse the cache flush process if you are going to read
> data written by the GPU.
> 
> 1) Make sure GPU is finished writing
> 2) flush your cache
> 3) read AGP memory like normal RAM.

We call that kind of flush an "invalidate" in fact :)

Ben.

Re: radeon, apertures & memory mapping

[Paul Mackerras]

Jon Smirl writes:

> > It works, but it's illegal. That means that the CPU might well speculate
> > a load from one of these pages in kernel-land just because it happens to
> > be next to a page where you are iterating an array, and may then bring a
> > bit in the cache from that page.
> 
> That shouldn't matter the page brought in would be for a speculative
> read and never accessed. It should just fall out of the cache and not
> be written back. There is only one cachable mapping. In this model
> writes are always followed by a flush before telling the GPU to access
> the memory that has just been written.

That would be fine, but it would mean making sure that every time any
code in the DRI, DRM or X server writes to the AGP memory, it does the
flush as well.  Sounds like a maintenance nightmare to me...

Paul.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
--

Re: radeon, apertures & memory mapping

[Ville Syrjälä]

On Mon, Mar 14, 2005 at 11:41:23AM +1100, Paul Mackerras wrote:
> Jon Smirl writes:
> 
> > > It works, but it's illegal. That means that the CPU might well speculate
> > > a load from one of these pages in kernel-land just because it happens to
> > > be next to a page where you are iterating an array, and may then bring a
> > > bit in the cache from that page.
> > 
> > That shouldn't matter the page brought in would be for a speculative
> > read and never accessed. It should just fall out of the cache and not
> > be written back. There is only one cachable mapping. In this model
> > writes are always followed by a flush before telling the GPU to access
> > the memory that has just been written.
> 
> That would be fine, but it would mean making sure that every time any
> code in the DRI, DRM or X server writes to the AGP memory, it does the
> flush as well.  Sounds like a maintenance nightmare to me...

It should be the responsibility of the memory manager. If anything wants 
to access the memory it would call lock() and when it's done with the 
memory it calls unlock(). That's exactly how DirectFB's memory manager 
works.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

> It should be the responsibility of the memory manager. If anything wants 
> to access the memory it would call lock() and when it's done with the 
> memory it calls unlock(). That's exactly how DirectFB's memory manager 
> works.

In an ideal world ... However, since we are planning to move the memory
manager to the kernel, that would mean a kernel access (syscall, ioctl,
whatever...) twice per access to AGP memory. Not realistic.

The case of the CP ring is easy to deal with by the macros we have there
already and it would be kernel-kernel. But it would be a hit for a lot
of other things I suppose.

Ben.

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Jon Smirl]

On Mon, 14 Mar 2005 12:05:59 +1100, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> 
> > It should be the responsibility of the memory manager. If anything wants
> > to access the memory it would call lock() and when it's done with the
> > memory it calls unlock(). That's exactly how DirectFB's memory manager
> > works.
> 
> In an ideal world ... However, since we are planning to move the memory
> manager to the kernel, that would mean a kernel access (syscall, ioctl,
> whatever...) twice per access to AGP memory. Not realistic.

I'm only suggesting this for the DRM/fbdev stack. Anything else from
user space can use a non-cached mapping.

It shouldn't hurt to have a parallel non-cached mapping being used in
conjuction with this protocol. By definition the non-cached mapping
never gets into an inconsistent state.

> 
> The case of the CP ring is easy to deal with by the macros we have there
> already and it would be kernel-kernel. But it would be a hit for a lot
> of other things I suppose.

The performance trade off is, how long does the invalidate take?  If
the CPU has 2MB of unflushed write data the instruction is going to
take a while to finish. In the non-cached scheme this data is flushed
in parallel with us playing with the AGP memory.  To flush 2MB takes
something like 2MB / 400Mhz * 64bytes * 2 (DDR) = 20 microseconds but
it may be more like 1 microsecond on average.

Thinking about this for a while you can't compute which is the better
strategy because everything depends on the workload and how dirty the
cache is. Best thing to do would be to code it up and try it. But I
want to get a dual head radeon driver working first.

It may also be true that the CP Ring is better left non-cached and
only access to the graphics buffers be done with the caching scheme.

BTW, you can implement super fast texture load/unload using a similar
scheme. Start with the texture in the user space program. Program
wants to upload the texture. Flush CPU cache. Point the GART at the
physical pages allocated to the user holding the texture. Now walk the
user's page table and mark those pages copy on write. Free the memory
the pages the GART was originally pointing at. Reverse the scheme to
get data from the GPU. For small textures it is faster to copy them
but if you are moving 20MB of data this is much faster.

-- 
Jon Smirl
jonsmirl@gmail.com

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

On Sun, 2005-03-13 at 20:47 -0500, Jon Smirl wrote:
> On Mon, 14 Mar 2005 12:05:59 +1100, Benjamin Herrenschmidt
> <benh@kernel.crashing.org> wrote:
> > 
> > > It should be the responsibility of the memory manager. If anything wants
> > > to access the memory it would call lock() and when it's done with the
> > > memory it calls unlock(). That's exactly how DirectFB's memory manager
> > > works.
> > 
> > In an ideal world ... However, since we are planning to move the memory
> > manager to the kernel, that would mean a kernel access (syscall, ioctl,
> > whatever...) twice per access to AGP memory. Not realistic.
> 
> I'm only suggesting this for the DRM/fbdev stack. Anything else from
> user space can use a non-cached mapping.

Then I don't see the point. Especially since the problem I explained
would still be there as long as there is a non-cached mapping.

> It shouldn't hurt to have a parallel non-cached mapping being used in
> conjuction with this protocol. By definition the non-cached mapping
> never gets into an inconsistent state.

Wrong :) It can badly conflict with the existence of a cached mapping.
Re-read my mail that explains the problem carefully.
 
> > The case of the CP ring is easy to deal with by the macros we have there
> > already and it would be kernel-kernel. But it would be a hit for a lot
> > of other things I suppose.
> 
> The performance trade off is, how long does the invalidate take?  If
> the CPU has 2MB of unflushed write data the instruction is going to
> take a while to finish. In the non-cached scheme this data is flushed
> in parallel with us playing with the AGP memory.  To flush 2MB takes
> something like 2MB / 400Mhz * 64bytes * 2 (DDR) = 20 microseconds but
> it may be more like 1 microsecond on average.
> 
> Thinking about this for a while you can't compute which is the better
> strategy because everything depends on the workload and how dirty the
> cache is. Best thing to do would be to code it up and try it. But I
> want to get a dual head radeon driver working first.
> 
> It may also be true that the CP Ring is better left non-cached and
> only access to the graphics buffers be done with the caching scheme.

Using write-through cache might be an interesting tradeoff

> BTW, you can implement super fast texture load/unload using a similar
> scheme. Start with the texture in the user space program. Program
> wants to upload the texture. Flush CPU cache. Point the GART at the
> physical pages allocated to the user holding the texture. Now walk the
> user's page table and mark those pages copy on write. Free the memory
> the pages the GART was originally pointing at. Reverse the scheme to
> get data from the GPU. For small textures it is faster to copy them
> but if you are moving 20MB of data this is much faster.
> 
-- 
Benjamin Herrenschmidt <benh@kernel.crashing.org>

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Paul Mackerras]

xorg@lists.freedesktop.org removed from CC since I can't post to it.

Jon Smirl writes:

> It shouldn't hurt to have a parallel non-cached mapping being used in
> conjuction with this protocol. By definition the non-cached mapping
> never gets into an inconsistent state.

According to the PowerPC Architecture specification, it is a
programming error to have both cacheable and uncacheable mappings of
the same page.  That means the hardware designers consider that they
don't have to worry if the hardware misbehaves if software does
that. :P  So that is not a feasible solution for us.

Paul.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
--

Re: Re: radeon, apertures & memory mapping

[Jon Smirl]

On Mon, 14 Mar 2005 15:07:26 +1100, Paul Mackerras <paulus@samba.org> wrote:
> xorg@lists.freedesktop.org removed from CC since I can't post to it.
> 
> Jon Smirl writes:
> 
> > It shouldn't hurt to have a parallel non-cached mapping being used in
> > conjuction with this protocol. By definition the non-cached mapping
> > never gets into an inconsistent state.
> 
> According to the PowerPC Architecture specification, it is a
> programming error to have both cacheable and uncacheable mappings of
> the same page.  That means the hardware designers consider that they
> don't have to worry if the hardware misbehaves if software does
> that. :P  So that is not a feasible solution for us.
> 
> Paul.

Ok, I see this is a problem for the PPC. I've never used a PPC so you
guys have to tell me what is illegal on it.

-- 
Jon Smirl
jonsmirl@gmail.com


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

On Sun, 2005-03-13 at 23:40 -0500, Jon Smirl wrote:
> On Mon, 14 Mar 2005 15:07:26 +1100, Paul Mackerras <paulus@samba.org> wrote:
> > xorg@lists.freedesktop.org removed from CC since I can't post to it.
> > 
> > Jon Smirl writes:
> > 
> > > It shouldn't hurt to have a parallel non-cached mapping being used in
> > > conjuction with this protocol. By definition the non-cached mapping
> > > never gets into an inconsistent state.
> > 
> > According to the PowerPC Architecture specification, it is a
> > programming error to have both cacheable and uncacheable mappings of
> > the same page.  That means the hardware designers consider that they
> > don't have to worry if the hardware misbehaves if software does
> > that. :P  So that is not a feasible solution for us.
> > 
> > Paul.
> 
> Ok, I see this is a problem for the PPC. I've never used a PPC so you
> guys have to tell me what is illegal on it.

And probably for other platforms as well. I'm pretty sure some Athlons
will be very upset too. I'm not even sure you can do that sort of tricks
on MIPS which has strange mapping rules, etc, etc, etc...

Anyway, mixing cacheable and non-cacheable mappings is asking for
trouble, just don't do it.

Having the ability to do both (selected by the platform type, or some
AGP errata bit, or whatever) is a different issue and might be worth
investigating.

Ben.




-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
--

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Soeren Sandmann]

Benjamin Herrenschmidt <benh@kernel.crashing.org> writes:

> In an ideal world ... However, since we are planning to move the memory
> manager to the kernel, that would mean a kernel access (syscall, ioctl,
> whatever...) twice per access to AGP memory. Not realistic.

Could the user space driver batch many such accesses together and use
a lock_many()/unlock_many() API?


Søren

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Ville Syrjälä]

On Mon, Mar 14, 2005 at 05:30:04PM +0100, Soeren Sandmann wrote:
> Benjamin Herrenschmidt <benh@kernel.crashing.org> writes:
> 
> > In an ideal world ... However, since we are planning to move the memory
> > manager to the kernel, that would mean a kernel access (syscall, ioctl,
> > whatever...) twice per access to AGP memory. Not realistic.
> 
> Could the user space driver batch many such accesses together and use
> a lock_many()/unlock_many() API?

Natrually it should try to do as much as possible during the 
lock()/unlock() sequence.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: [Linux-fbdev-devel] Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

On Mon, 2005-03-14 at 17:30 +0100, Soeren Sandmann wrote:
> Benjamin Herrenschmidt <benh@kernel.crashing.org> writes:
> 
> > In an ideal world ... However, since we are planning to move the memory
> > manager to the kernel, that would mean a kernel access (syscall, ioctl,
> > whatever...) twice per access to AGP memory. Not realistic.
> 
> Could the user space driver batch many such accesses together and use
> a lock_many()/unlock_many() API?

We may have to use a lock/unlock API anyway due to interaction with the
VGA arbiter in fact. If for some reason, the card can't completely
disable decoding of VGA and IO space, it needs to bracket any access to
the framebuffer with something.

That is unuseable for things like MOL though. We are giving the
framebuffer to some foreign OS in an emulation shell that doesn't know
how to do but blit directly at any time. Oh well, I need to think a bit
more about those sceniario.

Ben.

Re: radeon, apertures & memory mapping

[Ville Syrjälä]

On Sun, Mar 13, 2005 at 11:04:59PM +1100, Benjamin Herrenschmidt wrote:
> > 
> > I must be missing something something obvious because I don't quite 
> > understand what major drawbacks there are with the non-overlapping mode. 
> > As I see it you get at least the same amount of CPU accessible memory as 
> > you get in the overlapping mode.
> 
> Yes, you do, but that means that if the apertures are configured such
> that the entire VRAM fits in a single aperture, then you just can't use
> the second aperture at all. Which means you can't have separate swapper
> setting for both apertures, and thus, can't let two independant
> processes access the video memory with different bit depth, at least on
> big endian machines unless you do trickery, and play with the swapper
> before each access.

Ok so the problem is byte swapping. Looking at atyfb for example it uses 
the "big-endian" aperture on big-endian systems and selects the byte 
swapping method according to the bit depth. If that really means that all 
host access to the aperture gets byte swapped then I don't see how the 
current situation can work correctly for DirectFB. Offscreen surfaces can 
use any bit depth and so their bytes could be swapped incorrectly. Makes 
me wish I had a PPC box alongside the x86 one.

-- 
Ville Syrjälä
syrjala@sci.fi
http://www.sci.fi/~syrjala/

Re: radeon, apertures & memory mapping

[Donnie Berkholz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ville Syrjälä wrote:
> Makes me wish I had a PPC box alongside the x86 one.

You might like to try http://pearpc.sourceforge.net/.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFCNgGnXVaO67S1rtsRAuQ+AKCSkimy2poypyjls7ihX2bgtU6CygCfZJWy
aV1iW/PU2lKhe9MleB5+WyE=
=wEq+
-----END PGP SIGNATURE-----

Re: radeon, apertures & memory mapping

[Benjamin Herrenschmidt]

> 
> Ok so the problem is byte swapping. Looking at atyfb for example it uses 
> the "big-endian" aperture on big-endian systems and selects the byte 
> swapping method according to the bit depth. If that really means that all 
> host access to the aperture gets byte swapped then I don't see how the 
> current situation can work correctly for DirectFB. Offscreen surfaces can 
> use any bit depth and so their bytes could be swapped incorrectly. Makes 
> me wish I had a PPC box alongside the x86 one.

Possibly yes. In X, when blitting to an overlay surface, we
save/clear/restore the swapper indeed.

I would really like to keep the swappers independant though. Take things
like MOL (MacOnLinux). I maps the framebuffer and passes it to MacOS,
which expects proper swapping and doesn't (for a basic non-accelerated
framebuffer, which is what MOL implements) provide an easy way to
set/restore the swapper.

Also, doing so would require arbitration between clients using both
heads, while 2 independant swappers mean that even if the client wants
to temporarily disable it (for uploading an offscreen surface for
example), it has it's own swapper and won't conflict with whoever is
using the other framebuffer.

That means some knowledge about those swappers of course.

Note that almost all fbdev's on big endian platforms have some kind of
swapping mecanism on accesses, so that's something you'll have to deal
with anyway.

Ben.