[PATCH 0/1] Fix for BUG782: vnc server issue in mips

[PATCH 0/1] Fix for BUG782: vnc server issue in mips

[Yu Ke]

Bug 782 is that x86 vnc client does not work with vnc server in qemumips.

the root cause is that x11vnc server's word endian is wrong in qemumips.

mips linux is bigendian, while x86 linux is little endian, so there should be
endian convention between the vnc client/server. the code does have the endian
convertion logic and the logic works well in most case. the only issue is that
in vncserver mips compiling, there is missing flag "-DLIBVNCSERVER_WORDS_BIGENDIAN",
thus the vncserver in mips has wrong endian.

Adding the flag can fix this issue.

Pull URL: git://git.pokylinux.org/poky-contrib.git
  Branch: kyu3/bug782-vnc
  Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kyu3/bug782-vnc

Thanks,
    Yu Ke <ke.yu@intel.com>
---


Yu Ke (1):
  x11vnc: add big endian CFLAG for mips, to fix bug 782

 meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

[PATCH 1/1] x11vnc: add big endian CFLAG for mips, to fix bug 782

[Yu Ke]

linux mips is big endian, so need "LIBVNCSERVER_WORDS_BIGENDIAN"
flag for mips compilling.

this patch fix bug [YOCTO #782]

Signed-off-by: Yu Ke <ke.yu@intel.com>
---
 meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb b/meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb
index f5a6662..17e9593 100644
--- a/meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb
+++ b/meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb
@@ -17,3 +17,4 @@ SRC_URI[sha256sum] = "60a7cceee2c9a5f1c854340b2bae13f975ac55906237042f81f795b28a
 
 inherit autotools
 
+CFLAGS_mips += "-DLIBVNCSERVER_WORDS_BIGENDIAN"
-- 
1.7.0.4

Re: [PATCH 1/1] x11vnc: add big endian CFLAG for mips, to fix bug 782

[Khem Raj]

On 3/3/2011 7:50 PM, Yu Ke wrote:
> linux mips is big endian, so need "LIBVNCSERVER_WORDS_BIGENDIAN"
> flag for mips compilling.
>
> this patch fix bug [YOCTO #782]
>
> Signed-off-by: Yu Ke<ke.yu@intel.com>
> ---
>   meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb |    1 +
>   1 files changed, 1 insertions(+), 0 deletions(-)
>
> diff --git a/meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb b/meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb
> index f5a6662..17e9593 100644
> --- a/meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb
> +++ b/meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb
> @@ -17,3 +17,4 @@ SRC_URI[sha256sum] = "60a7cceee2c9a5f1c854340b2bae13f975ac55906237042f81f795b28a
>
>   inherit autotools
>
> +CFLAGS_mips += "-DLIBVNCSERVER_WORDS_BIGENDIAN"

I see that this package uses autconf macro AC_C_BIGENDIAN
to assert endianness. I would interested to know why this test does not 
work.

Re: [PATCH 1/1] x11vnc: add big endian CFLAG for mips, to fix bug 782

[Yu Ke]

On Mar 03, 21:11, Khem Raj wrote:
> On 3/3/2011 7:50 PM, Yu Ke wrote:
> >linux mips is big endian, so need "LIBVNCSERVER_WORDS_BIGENDIAN"
> >flag for mips compilling.
> >
> >this patch fix bug [YOCTO #782]
> >
> >Signed-off-by: Yu Ke<ke.yu@intel.com>
> >---
> >  meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb |    1 +
> >  1 files changed, 1 insertions(+), 0 deletions(-)
> >
> >diff --git a/meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb b/meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb
> >index f5a6662..17e9593 100644
> >--- a/meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb
> >+++ b/meta/recipes-graphics/x11vnc/x11vnc_0.9.12.bb
> >@@ -17,3 +17,4 @@ SRC_URI[sha256sum] = "60a7cceee2c9a5f1c854340b2bae13f975ac55906237042f81f795b28a
> >
> >  inherit autotools
> >
> >+CFLAGS_mips += "-DLIBVNCSERVER_WORDS_BIGENDIAN"
> 
> I see that this package uses autconf macro AC_C_BIGENDIAN
> to assert endianness. I would interested to know why this test does
> not work.

Good observation. I take more look, and find although AC_C_BIGENDIAN is used in autocof, the src code unfortunately did not use the generated WORDS_BIGENDIAN, instead, it use LIBVNCSERVER_WORDS_BIGENDIAN, so that is the reason why AC_C_BIGENDIAN has no effect.

in the case, the more elegant fix seems replacing LIBVNCSERVER_WORDS_BIGENDIAN with WORDS_BIGENDIAN. i will have a try.

Regards
Ke

> _______________________________________________
> poky mailing list
> poky@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/poky