Mirror "module" for framebuffer

Mirror "module" for framebuffer

[Daniele Salvatore Albano]

Hi to all,

In short i should start to work to an application to remote control an 
android phone, just using the browser (video stream plus a lot of 
javascript).
While this is partially working right now, as you can imagine, phone is 
slow, a lot slow, so, while the framebuffer is little in most cases 
(320x256x2/320x256x4/800x480x2/800x480x4) a simple comparison, using 
neon extension too, if avaiable, is too slow, so the screen is refreshed 
too few times in a sec.

On windows exists Mirror Drivers infrastructure that supply a list of 
changed parts of the screen and more (mouse events too if i'm not 
wrong), but looking around on the web i didn't founded anything related 
to this stuff (or similar) for linux.

So, before i start scrambling docs and source code (i've really little 
experience with kernel module programming, i've done simply things), i 
want to ask a simple question: can be done using a sort of hooking of 
internal framebuffer function calls (fb_ops for example) or it's 
impossible because apps write directly into the framebuffer memory?

Thank you!!!


Best Regards,
Daniele

-- 
____________________________________________________________
                                         |
Daniele Salvatore Albano                | web site:
  IT Consultant                          | http://www.itechcon.it
  Website Design and Development         |
  Software Engineer and Developer        | e-mail:
  Linux Servers SetUp And Administration | info@itechcon.it
  Embedded Network Solutions             | d.albano@itechcon.it

Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni contenute in 
questo messaggio sono riservate ed a uso esclusivo del destinatario. 
Qualora il messaggio in parola Le fosse pervenuto per errore, La 
invitiamo ad eliminarlo senza copiarlo e a non inoltrarlo a terzi, 
dandocene gentilmente comunicazione. Grazie.
Pursuant to Legislative Decree No. 196/2003, you are hereby informed 
that this message contains confidential information intended only for 
the use of the addressee. If you are not the addressee, and have 
received this message by mistake, please delete it and immediately 
notify us. You may not copy or disseminate this message to anyone. Thank 
you.

Re: Mirror "module" for framebuffer

[Bruno Prémont]

Hi Daniele,

On Wed, 01 June 2011 Daniele Salvatore Albano <info@daccii.it> wrote:
> In short i should start to work to an application to remote control an 
> android phone, just using the browser (video stream plus a lot of 
> javascript).
> While this is partially working right now, as you can imagine, phone is 
> slow, a lot slow, so, while the framebuffer is little in most cases 
> (320x256x2/320x256x4/800x480x2/800x480x4) a simple comparison, using 
> neon extension too, if avaiable, is too slow, so the screen is refreshed 
> too few times in a sec.
> 
> On windows exists Mirror Drivers infrastructure that supply a list of 
> changed parts of the screen and more (mouse events too if i'm not 
> wrong), but looking around on the web i didn't founded anything related 
> to this stuff (or similar) for linux.
> 
> So, before i start scrambling docs and source code (i've really little 
> experience with kernel module programming, i've done simply things), i 
> want to ask a simple question: can be done using a sort of hooking of 
> internal framebuffer function calls (fb_ops for example) or it's 
> impossible because apps write directly into the framebuffer memory?

fbdefio probably is the nearest to what you are looking for.

It catches changes to the framebuffer (at page level) and calls back to
driver at regular intervals to react on those changes.

Though if your Andriod has some video HW acceleration you are going to
be lost here unless you can get change notification from the hardware
itself and then read back the changes.

Bruno

Re: Mirror "module" for framebuffer

[Bruno Prémont]

Hi,

[re-adding linux-fbdev, please make sure to reply-to-all]

On Wed, 01 June 2011 Daniele Salvatore Albano <info@daccii.it> wrote:
>  From what i understand, technically i should hook registered 
> framebuffer and change the callback (info->fbdefio->deferred_io), 
> restoring it on module unloading to avoid the caos.
>
> In this way i could "catch" updates passed to the pagelist argument of 
> the deferred io callback, add them to an own list, and call the original 
> callback to let the driver to do the own work.

Note that some framebuffer drivers might implement (lots) of private
ioctls to implement (accelerated) drawing primitives.
So unless you know the driver you are hooking onto all bets are open.

> This is in the kernel from 2.6.20, so it should be on phone kernel.
> 
> I've a couple more questions:
> - i need to access the structure containing the list of registered 
> framebuffer device (struct fb_info *registered_fb[FB_MAX]), i've seen 
> that there is an export symbol on registered_fb so can i access that 
> directly/num_registered_fb?

You can access it (as does fbcon) though that direct access may/will
go away in the future as (since 2.6.39) now that array is protected
by a mutex on fb core side (fbcon being indirectly fine through its
synchronization via notification events, except when loading fbcon
module)
If you do access the list, write an function to do just that
so there is no big rewrite needed to move your driver forward.

> - i need to know when a framebuffer is registered, i've seen that there 
> is fb_register_client ... from what i see it seems that callback are 
> called on every event so from the callback itself i should check for 
> FB_EVENT_FB_REGISTERED, is this right?

If you want to know when framebuffers get (un)registered (or get
reconfigured) this is the way to go.

> - (stupid question) do all fb modules implement deferred input/output?

No, only a few selected ones implement deferred input/output, usually
because the real framebuffer cannot be mapped into memory or the link
to it is just dead slow so changes are pushed down to device at regular
intervals.


I'm wondering if you not better had to create a new framebuffer driver
(based on virtual framebuffer) with fbdefio to push the changes to a
backing framebuffer [the way fbcon draws console] (as well as whatever you
use to stream to the remote system).
This will make you lose any potential hardware acceleration but keep
things simple.
Though you would need ability to move software to your new framebuffer.

Bruno

Re: Mirror "module" for framebuffer

[Daniele Salvatore Albano]

Il 01/06/2011 22:02, Bruno Prémont ha scritto:
> Hi,
>
> [re-adding linux-fbdev, please make sure to reply-to-all]

uops

 > Note that some framebuffer drivers might implement (lots) of private
 > ioctls to implement (accelerated) drawing primitives.
 > So unless you know the driver you are hooking onto all bets are open.
.
.
.
> No, only a few selected ones implement deferred input/output, usually
> because the real framebuffer cannot be mapped into memory or the link
> to it is just dead slow so changes are pushed down to device at regular
> intervals.
.
.
.
> I'm wondering if you not better had to create a new framebuffer driver
> (based on virtual framebuffer) with fbdefio to push the changes to a
> backing framebuffer [the way fbcon draws console] (as well as whatever you
> use to stream to the remote system).
> This will make you lose any potential hardware acceleration but keep
> things simple.
> Though you would need ability to move software to your new framebuffer.

Main problem is that i need to show what is being displayed on the 
screen, so i can't use another framebuffer device but i must use the one 
used by the os.

mmm i think that isin't the right way to handle this situation because 
this software should work with too much different (and untestable) hardware.
I'll see if there is a way to check when android draw something on the 
framebuffer

Thanks a lot for your time!


Best Regards,
Daniele
-- 
____________________________________________________________
                                         |
Daniele Salvatore Albano                | web site:
  IT Consultant                          | http://www.itechcon.it
  Website Design and Development         |
  Software Engineer and Developer        | e-mail:
  Linux Servers SetUp And Administration | info@itechcon.it
  Embedded Network Solutions             | d.albano@itechcon.it

Are you lazy? Take a look to LazyDroid (www.lazydroid.net): android for 
lazy people!

Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni contenute in 
questo messaggio sono riservate ed a uso esclusivo del destinatario. 
Qualora il messaggio in parola Le fosse pervenuto per errore, La 
invitiamo ad eliminarlo senza copiarlo e a non inoltrarlo a terzi, 
dandocene gentilmente comunicazione. Grazie.
Pursuant to Legislative Decree No. 196/2003, you are hereby informed 
that this message contains confidential information intended only for 
the use of the addressee. If you are not the addressee, and have 
received this message by mistake, please delete it and immediately 
notify us. You may not copy or disseminate this message to anyone. Thank 
you.