dma-buf/fbdev: one-to-many support

dma-buf/fbdev: one-to-many support

[David Herrmann]

Hi

I am currently working on fblog [1] (a replacement for fbcon without
VT dependencies) but this questions does also apply to other fbdev
users. Is there a way to share framebuffers between fbdev devices? I
was thinking especially of USB devices like DisplayLink. If they share
the same screen dimensions it would increase performance a lot if I
could display a single buffer on all the devices instead of copying it
into each framebuffer.

I was told to have a look at the dma-buf framework to implement this.
However, looking at the fbdev dma-buf support I think that this isn't
currently possible. Each fbdev device takes the exporter-role and
provides a single dma-buf object. However, if I wanted to share the
buffers, I would need to be the exporter. Or there needs to be a way
for the fbdev devices to import a dma-buf from other fbdev devices.

I also took a short look at DRM prime support and noticed that it is
capable of importing buffers (or at least it looks like it is).
Therefore,  I was wondering whether it does make sense to add an
"import dma-buf" callback to fbdev devices and if the fbdev driver
supports this, I can simply draw to a single dma-buf from one fbdev
device and push it to all other fbdev devices that share the same
dimensions.
It would also be nice to allow multiple buffer-owners or a way to
transfer ownership. That is, if the owner/exporter of the dma-buf
vanishes, I would pass it to another fbdev device which would pick it
up so I don't have to create a new one.

I think this is only interesting for DisplayLink-devices as they are
currently the only way to get a bunch of displays connected to a
single machine. Anyway, if you think that this isn't worth it, I will
probably drop this idea.

Regards
David

[1] fblog kernel driver: http://lwn.net/Articles/505965/

Re: dma-buf/fbdev: one-to-many support

[Laurent Pinchart]

Hi David,

On Saturday 14 July 2012 16:10:56 David Herrmann wrote:
> Hi
> 
> I am currently working on fblog [1] (a replacement for fbcon without
> VT dependencies) but this questions does also apply to other fbdev
> users. Is there a way to share framebuffers between fbdev devices? I
> was thinking especially of USB devices like DisplayLink. If they share
> the same screen dimensions it would increase performance a lot if I
> could display a single buffer on all the devices instead of copying it
> into each framebuffer.
> 
> I was told to have a look at the dma-buf framework to implement this.
> However, looking at the fbdev dma-buf support I think that this isn't
> currently possible. Each fbdev device takes the exporter-role and
> provides a single dma-buf object. However, if I wanted to share the
> buffers, I would need to be the exporter. Or there needs to be a way
> for the fbdev devices to import a dma-buf from other fbdev devices.
> 
> I also took a short look at DRM prime support and noticed that it is
> capable of importing buffers (or at least it looks like it is).
> Therefore,  I was wondering whether it does make sense to add an
> "import dma-buf" callback to fbdev devices and if the fbdev driver
> supports this, I can simply draw to a single dma-buf from one fbdev
> device and push it to all other fbdev devices that share the same
> dimensions.

The main issue is that fbdev has been designed with the implicit assumption 
that an fbdev driver will always own the graphics memory it uses. All 
components in the stack, from drivers to applications, have been designed 
around that assumption.

We could of course fix this, revamp the fbdev API and turn it into a modern 
graphics API, but I really wonder whether it would be worth it. DRM has been 
getting quite a lot of attention lately, especially from embedded developers 
and vendors, and the trend seems to me like the (Linux) world will gradually 
move from fbdev to DRM.

Please feel free to disagree :-)

> It would also be nice to allow multiple buffer-owners or a way to
> transfer ownership. That is, if the owner/exporter of the dma-buf
> vanishes, I would pass it to another fbdev device which would pick it
> up so I don't have to create a new one.
> 
> I think this is only interesting for DisplayLink-devices as they are
> currently the only way to get a bunch of displays connected to a
> single machine. Anyway, if you think that this isn't worth it, I will
> probably drop this idea.
> 
> Regards
> David
> 
> [1] fblog kernel driver: http://lwn.net/Articles/505965/

-- 
Regards,

Laurent Pinchart

Re: dma-buf/fbdev: one-to-many support

[Alan Cox]

> The main issue is that fbdev has been designed with the implicit assumption 
> that an fbdev driver will always own the graphics memory it uses. All 
> components in the stack, from drivers to applications, have been designed 
> around that assumption.
> 
> We could of course fix this, revamp the fbdev API and turn it into a modern 
> graphics API, but I really wonder whether it would be worth it. DRM has been 
> getting quite a lot of attention lately, especially from embedded developers 
> and vendors, and the trend seems to me like the (Linux) world will gradually 
> move from fbdev to DRM.
> 
> Please feel free to disagree :-)

I would disagree on the "main issue" bit. All the graphics cards have
their own formats and cache management rules. Simply sharing a buffer
doesn't work - which is why all of the extra gloop will be needed.

Alan

Re: dma-buf/fbdev: one-to-many support

[David Herrmann]

Hi Laurent and Alan

On Tue, Jul 17, 2012 at 1:24 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> The main issue is that fbdev has been designed with the implicit assumption
>> that an fbdev driver will always own the graphics memory it uses. All
>> components in the stack, from drivers to applications, have been designed
>> around that assumption.
>>
>> We could of course fix this, revamp the fbdev API and turn it into a modern
>> graphics API, but I really wonder whether it would be worth it. DRM has been
>> getting quite a lot of attention lately, especially from embedded developers
>> and vendors, and the trend seems to me like the (Linux) world will gradually
>> move from fbdev to DRM.
>>
>> Please feel free to disagree :-)
>
> I would disagree on the "main issue" bit. All the graphics cards have
> their own formats and cache management rules. Simply sharing a buffer
> doesn't work - which is why all of the extra gloop will be needed.

This is exactly why I suggested adding an "owner" field. A driver
could then check whether the buffer it is supposed to share/takeover
is from a compatible (or even the same) driver/device. If it is not,
it would simply reject using the buffer. Then again, if we have
multiple devices that are incompatible, we are still unable to share
the buffer. So this attempt would only be useful if we have tons of
DisplayLink devices attached that all use the same driver, for
example.

Regarding DRM: In user-space I prefer DRM over fbdev. With the
introduction of the dumb-buffers there isn't even the need to have
mesa installed. However, fblog runs in kernel space and currently
cannot use DRM as there is no in-kernel DRM API. I looked at
drm-fops.c whether it is easy to create a very simple in-kernel API
but then I dropped the idea as this might be too complex for a simple
debugging-only driver. Another attempt would be making the
drm-fb-helper more generic so we can use this layer as in-kernel DRM
API.

I had a deeper look into this this weekend and so as a summary I think
all in-kernel graphics access is probably not worth optimizing it.
fbcon is already working great and fblog is only used during boot and
oopses/panics and can be restricted to a single device. I will have
another look at the drivers in a few weeks but if you tell me that
this is not easy to implement, I will probably have to let this idea
go.

Thanks
David

Re: dma-buf/fbdev: one-to-many support

[Laurent Pinchart]

Hi David,

On Tuesday 17 July 2012 14:23:18 David Herrmann wrote:
> On Tue, Jul 17, 2012 at 1:24 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> >> The main issue is that fbdev has been designed with the implicit
> >> assumption that an fbdev driver will always own the graphics memory it
> >> uses. All components in the stack, from drivers to applications, have
> >> been designed around that assumption.
> >> 
> >> We could of course fix this, revamp the fbdev API and turn it into a
> >> modern graphics API, but I really wonder whether it would be worth it.
> >> DRM has been getting quite a lot of attention lately, especially from
> >> embedded developers and vendors, and the trend seems to me like the
> >> (Linux) world will gradually move from fbdev to DRM.
> >> 
> >> Please feel free to disagree :-)
> > 
> > I would disagree on the "main issue" bit. All the graphics cards have
> > their own formats and cache management rules. Simply sharing a buffer
> > doesn't work - which is why all of the extra gloop will be needed.
> 
> This is exactly why I suggested adding an "owner" field. A driver
> could then check whether the buffer it is supposed to share/takeover
> is from a compatible (or even the same) driver/device. If it is not,
> it would simply reject using the buffer. Then again, if we have
> multiple devices that are incompatible, we are still unable to share
> the buffer. So this attempt would only be useful if we have tons of
> DisplayLink devices attached that all use the same driver, for
> example.
> 
> Regarding DRM: In user-space I prefer DRM over fbdev. With the
> introduction of the dumb-buffers there isn't even the need to have
> mesa installed. However, fblog runs in kernel space and currently
> cannot use DRM as there is no in-kernel DRM API. I looked at
> drm-fops.c whether it is easy to create a very simple in-kernel API
> but then I dropped the idea as this might be too complex for a simple
> debugging-only driver. Another attempt would be making the
> drm-fb-helper more generic so we can use this layer as in-kernel DRM
> API.
> 
> I had a deeper look into this this weekend and so as a summary I think
> all in-kernel graphics access is probably not worth optimizing it.
> fbcon is already working great and fblog is only used during boot and
> oopses/panics and can be restricted to a single device. I will have
> another look at the drivers in a few weeks but if you tell me that
> this is not easy to implement, I will probably have to let this idea
> go.

My gut feeling is that, given the effort required to add new APIs, it would be 
more interesting to work on an in-kernel DRM API to make drmcon and drmlog 
implementations possible without any fbdev compatibility layer. That's rather 
a long term goal though.

-- 
Regards,

Laurent Pinchart