Replacing Web in Sato with Midori

Replacing Web in Sato with Midori

[Burton, Ross]

Hi,

As everyone who's used it can attest, Web (the optional browser in
Sato) is pretty rough.  Part of my plans about replacing Sato with a
leaner environment involves replacing it with Midori, and if there
isn't any disagreements I'll work on a submission to merge Midori into
Sato now for everyone who expects the Sato web browser to be useful.

This will involve pulling a few projects from meta-oe to oe-core:
ca-certificates, python-docutils and vala specifically (although its
possible that we can drop the vala dependency).

Ross

Re: Replacing Web in Sato with Midori

[Koen Kooi]

Op 8 aug. 2012, om 10:41 heeft "Burton, Ross" <ross.burton@intel.com> het volgende geschreven:

> Hi,
> 
> As everyone who's used it can attest, Web (the optional browser in
> Sato) is pretty rough.  Part of my plans about replacing Sato with a
> leaner environment involves replacing it with Midori, and if there
> isn't any disagreements I'll work on a submission to merge Midori into
> Sato now for everyone who expects the Sato web browser to be useful.
> 
> This will involve pulling a few projects from meta-oe to oe-core:
> ca-certificates, python-docutils and vala specifically (although its
> possible that we can drop the vala dependency).

Adding more stuff to oe-core is a bad idea. You should take this opportunity to split all the sato stuff into its own layer.

Re: Replacing Web in Sato with Midori

[Richard Purdie]

On Wed, 2012-08-08 at 12:01 +0200, Koen Kooi wrote:
> Op 8 aug. 2012, om 10:41 heeft "Burton, Ross" <ross.burton@intel.com> het volgende geschreven:
> 
> > Hi,
> > 
> > As everyone who's used it can attest, Web (the optional browser in
> > Sato) is pretty rough.  Part of my plans about replacing Sato with a
> > leaner environment involves replacing it with Midori, and if there
> > isn't any disagreements I'll work on a submission to merge Midori into
> > Sato now for everyone who expects the Sato web browser to be useful.
> > 
> > This will involve pulling a few projects from meta-oe to oe-core:
> > ca-certificates, python-docutils and vala specifically (although its
> > possible that we can drop the vala dependency).
> 
> Adding more stuff to oe-core is a bad idea. You should take this
> opportunity to split all the sato stuff into its own layer.

I feel very strongly that having a core layer with no way of
demonstrating and testing it is a very bad idea. I haven't changed my
mind about this and am very unlikely to. "How do you know it works?" is
the question you ask about package upgrades for example.

I know Ross is planning to rationalise what is in sato and ultimately
replace it with something more suited to this purpose. I don't think
removing it entirely is a good move.

One of the steps involved here is to replace what is currently a rather
hacky browser with a real world usable one which fulfils the above
objective - allow OE-Core to be testable. "web" is a core technology and
important to a significant subset of devices. So some things need to get
added, some will also get removed and things will become much less
"sato" like over time.

For example, pimlico is likely just to get removed/moved out to
meta-gnome or wherever as PIM isn't core technology like web is.

So I'd like to give Ross some room to manoeuvre and try to improve this.
It will involve some additions, deletions will also happen. I for one
think this proposal is a good first step (of many).

Cheers,

Richard

Re: Replacing Web in Sato with Midori

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1513 bytes --]

On Wed, Aug 08, 2012 at 01:23:08PM +0100, Richard Purdie wrote:
> On Wed, 2012-08-08 at 12:01 +0200, Koen Kooi wrote:
> > Op 8 aug. 2012, om 10:41 heeft "Burton, Ross" <ross.burton@intel.com> het volgende geschreven:
> > 
> > > Hi,
> > > 
> > > As everyone who's used it can attest, Web (the optional browser in
> > > Sato) is pretty rough.  Part of my plans about replacing Sato with a
> > > leaner environment involves replacing it with Midori, and if there
> > > isn't any disagreements I'll work on a submission to merge Midori into
> > > Sato now for everyone who expects the Sato web browser to be useful.
> > > 
> > > This will involve pulling a few projects from meta-oe to oe-core:
> > > ca-certificates, python-docutils and vala specifically (although its
> > > possible that we can drop the vala dependency).
> > 
> > Adding more stuff to oe-core is a bad idea. You should take this
> > opportunity to split all the sato stuff into its own layer.
> 
> I feel very strongly that having a core layer with no way of
> demonstrating and testing it is a very bad idea. I haven't changed my
> mind about this and am very unlikely to. "How do you know it works?" is
> the question you ask about package upgrades for example.

And does it need to be in the same layer?

Why not test webkit-gtk from oe-core with midori from meta-oe layer? 
Or meta-browser layer if meta-oe is too big for testing webkit-gtk.

Cheers,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Replacing Web in Sato with Midori

[Richard Purdie]

On Wed, 2012-08-08 at 14:36 +0200, Martin Jansa wrote:
> On Wed, Aug 08, 2012 at 01:23:08PM +0100, Richard Purdie wrote:
> > On Wed, 2012-08-08 at 12:01 +0200, Koen Kooi wrote:
> > > Op 8 aug. 2012, om 10:41 heeft "Burton, Ross" <ross.burton@intel.com> het volgende geschreven:
> > > 
> > > > Hi,
> > > > 
> > > > As everyone who's used it can attest, Web (the optional browser in
> > > > Sato) is pretty rough.  Part of my plans about replacing Sato with a
> > > > leaner environment involves replacing it with Midori, and if there
> > > > isn't any disagreements I'll work on a submission to merge Midori into
> > > > Sato now for everyone who expects the Sato web browser to be useful.
> > > > 
> > > > This will involve pulling a few projects from meta-oe to oe-core:
> > > > ca-certificates, python-docutils and vala specifically (although its
> > > > possible that we can drop the vala dependency).
> > > 
> > > Adding more stuff to oe-core is a bad idea. You should take this
> > > opportunity to split all the sato stuff into its own layer.
> > 
> > I feel very strongly that having a core layer with no way of
> > demonstrating and testing it is a very bad idea. I haven't changed my
> > mind about this and am very unlikely to. "How do you know it works?" is
> > the question you ask about package upgrades for example.
> 
> And does it need to be in the same layer?
> 
> Why not test webkit-gtk from oe-core with midori from meta-oe layer? 
> Or meta-browser layer if meta-oe is too big for testing webkit-gtk.

It needs to be in the same layer. The logistics of including sections of
meta-oe or meta-browser on the autobuilder whilst trying to test things
like "bitbake world" is not somewhere I want to go. The Yocto Project
does not have the manpower at this point in time to increase test
coverage to include meta-browser or meta-oe. If someone gives me access
to a large team of engineers...

Cheers,

Richard

Re: Replacing Web in Sato with Midori

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2662 bytes --]

On Wed, Aug 08, 2012 at 01:48:18PM +0100, Richard Purdie wrote:
> On Wed, 2012-08-08 at 14:36 +0200, Martin Jansa wrote:
> > On Wed, Aug 08, 2012 at 01:23:08PM +0100, Richard Purdie wrote:
> > > On Wed, 2012-08-08 at 12:01 +0200, Koen Kooi wrote:
> > > > Op 8 aug. 2012, om 10:41 heeft "Burton, Ross" <ross.burton@intel.com> het volgende geschreven:
> > > > 
> > > > > Hi,
> > > > > 
> > > > > As everyone who's used it can attest, Web (the optional browser in
> > > > > Sato) is pretty rough.  Part of my plans about replacing Sato with a
> > > > > leaner environment involves replacing it with Midori, and if there
> > > > > isn't any disagreements I'll work on a submission to merge Midori into
> > > > > Sato now for everyone who expects the Sato web browser to be useful.
> > > > > 
> > > > > This will involve pulling a few projects from meta-oe to oe-core:
> > > > > ca-certificates, python-docutils and vala specifically (although its
> > > > > possible that we can drop the vala dependency).
> > > > 
> > > > Adding more stuff to oe-core is a bad idea. You should take this
> > > > opportunity to split all the sato stuff into its own layer.
> > > 
> > > I feel very strongly that having a core layer with no way of
> > > demonstrating and testing it is a very bad idea. I haven't changed my
> > > mind about this and am very unlikely to. "How do you know it works?" is
> > > the question you ask about package upgrades for example.
> > 
> > And does it need to be in the same layer?
> > 
> > Why not test webkit-gtk from oe-core with midori from meta-oe layer? 
> > Or meta-browser layer if meta-oe is too big for testing webkit-gtk.
> 
> It needs to be in the same layer. The logistics of including sections of
> meta-oe or meta-browser on the autobuilder whilst trying to test things
> like "bitbake world" is not somewhere I want to go. The Yocto Project
> does not have the manpower at this point in time to increase test
> coverage to include meta-browser or meta-oe. If someone gives me access
> to a large team of engineers...

at least with webkit-efl/eve I usually see more issues in runtime then
build, so testing that "bitbake world" still builds something is very
far from "test coverage" for end users.

And to build only midori with meta-oe in another workspace with meta-oe
included (where you don't need to run "bitbake world") doesn't demand so
much more manpower. And someone should start midori probably in
some qemu* machine and try at least one http:// and https:// page to say 
that webkit-gtk is test covered.

Cheers,
-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Replacing Web in Sato with Midori

[Koen Kooi]

Op 8 aug. 2012, om 14:36 heeft Martin Jansa <martin.jansa@gmail.com> het volgende geschreven:

> On Wed, Aug 08, 2012 at 01:23:08PM +0100, Richard Purdie wrote:
>> On Wed, 2012-08-08 at 12:01 +0200, Koen Kooi wrote:
>>> Op 8 aug. 2012, om 10:41 heeft "Burton, Ross" <ross.burton@intel.com> het volgende geschreven:
>>> 
>>>> Hi,
>>>> 
>>>> As everyone who's used it can attest, Web (the optional browser in
>>>> Sato) is pretty rough.  Part of my plans about replacing Sato with a
>>>> leaner environment involves replacing it with Midori, and if there
>>>> isn't any disagreements I'll work on a submission to merge Midori into
>>>> Sato now for everyone who expects the Sato web browser to be useful.
>>>> 
>>>> This will involve pulling a few projects from meta-oe to oe-core:
>>>> ca-certificates, python-docutils and vala specifically (although its
>>>> possible that we can drop the vala dependency).
>>> 
>>> Adding more stuff to oe-core is a bad idea. You should take this
>>> opportunity to split all the sato stuff into its own layer.
>> 
>> I feel very strongly that having a core layer with no way of
>> demonstrating and testing it is a very bad idea. I haven't changed my
>> mind about this and am very unlikely to. "How do you know it works?" is
>> the question you ask about package upgrades for example.
> 
> And does it need to be in the same layer?
> 
> Why not test webkit-gtk from oe-core with midori from meta-oe layer? 
> Or meta-browser layer if meta-oe is too big for testing webkit-gtk.

Exactly, you can't make an argument against extra layers with a straight face, since oe-core is all about layers. There are a ton of recipes in oe-core that need extra layers to get properly tested (QT comes to mind), so I don't get why webkit (or by extension sato) should be so different.

Re: Replacing Web in Sato with Midori

[Paul Eggleton]

On Wednesday 08 August 2012 15:56:33 Koen Kooi wrote:
> There are a ton of recipes in oe-core that need extra layers to get properly
> tested (QT comes to mind),

What are you referring to? Qt should be perfectly testable within OE-Core.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Replacing Web in Sato with Midori

[Koen Kooi]

Op 8 aug. 2012, om 16:03 heeft Paul Eggleton <paul.eggleton@linux.intel.com> het volgende geschreven:

> On Wednesday 08 August 2012 15:56:33 Koen Kooi wrote:
>> There are a ton of recipes in oe-core that need extra layers to get properly
>> tested (QT comes to mind),
> 
> What are you referring to? Qt should be perfectly testable within OE-Core.

What kind of big qt apps does oe-core have to test it? I had to enable meta-kde to find bugs in the qt recipes, so if the tests are in there, they aren't get run properly.

Re: Replacing Web in Sato with Midori

[Paul Eggleton]

On Wednesday 08 August 2012 16:47:05 Koen Kooi wrote:
> Op 8 aug. 2012, om 16:03 heeft Paul Eggleton <paul.eggleton@linux.intel.com> 
het volgende geschreven:
> > On Wednesday 08 August 2012 15:56:33 Koen Kooi wrote:
> >> There are a ton of recipes in oe-core that need extra layers to get
> >> properly tested (QT comes to mind),
> > 
> > What are you referring to? Qt should be perfectly testable within OE-Core.
> 
> What kind of big qt apps does oe-core have to test it? I had to enable
> meta-kde to find bugs in the qt recipes, so if the tests are in there, they
> aren't get run properly.

And? KDE is far more demanding upon Qt than most Qt-based applications. We had 
to make specific changes to our Qt recipes just to accommodate it.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Replacing Web in Sato with Midori

[Samuel Stirtzel]

2012/8/8 Paul Eggleton <paul.eggleton@linux.intel.com>:
> On Wednesday 08 August 2012 16:47:05 Koen Kooi wrote:
>> Op 8 aug. 2012, om 16:03 heeft Paul Eggleton <paul.eggleton@linux.intel.com>
> het volgende geschreven:
>> > On Wednesday 08 August 2012 15:56:33 Koen Kooi wrote:
>> >> There are a ton of recipes in oe-core that need extra layers to get
>> >> properly tested (QT comes to mind),
>> >
>> > What are you referring to? Qt should be perfectly testable within OE-Core.
>>
>> What kind of big qt apps does oe-core have to test it? I had to enable
>> meta-kde to find bugs in the qt recipes, so if the tests are in there, they
>> aren't get run properly.
>
> And? KDE is far more demanding upon Qt than most Qt-based applications. We had
> to make specific changes to our Qt recipes just to accommodate it.

If you're talking about adding accessibility and session management to
configure -> these are set by default by Qt.
Don't know why they where unset in oe-core though.

>
> Cheers,
> Paul



-- 
Regards
Samuel

Re: Replacing Web in Sato with Midori

[Paul Eggleton]

On Wednesday 08 August 2012 17:04:16 Samuel Stirtzel wrote:
> > On Wednesday 08 August 2012 16:47:05 Koen Kooi wrote:
> >> Op 8 aug. 2012, om 16:03 heeft Paul Eggleton
> >> <paul.eggleton@linux.intel.com>> 
> > het volgende geschreven:
>
> > And? KDE is far more demanding upon Qt than most Qt-based applications. We
> > had to make specific changes to our Qt recipes just to accommodate it.
>
> If you're talking about adding accessibility and session management to
> configure -> these are set by default by Qt.
> Don't know why they where unset in oe-core though.

Not specifically, I was mainly referring to changes such as providing qt4-
native; we did not need to do that before.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Replacing Web in Sato with Midori

[Phil Blundell]

On Wed, 2012-08-08 at 09:41 +0100, Burton, Ross wrote:
> As everyone who's used it can attest, Web (the optional browser in
> Sato) is pretty rough.  Part of my plans about replacing Sato with a
> leaner environment involves replacing it with Midori, and if there
> isn't any disagreements I'll work on a submission to merge Midori into
> Sato now for everyone who expects the Sato web browser to be useful.

Replacing Web with Midori in Sato probably is a fine idea from the point
of view of those folks who want to use Sato per se.  As far as oe-core
is concerned, the point of having Sato included is apparently for
testability and it's not entirely obvious that much extra test coverage
would be gained by merging Midori.  

Indeed, it's not totally clear that having WebKit in meta-sato is really
justified by the test coverage it brings.  I think WebKit itself might
be a reasonable candidate for inclusion in oe-core proper, but the
current situation of having a slightly half-baked recipe in meta-sato is
not very satisfactory.

However...

>This will involve pulling a few projects from meta-oe to oe-core:
>ca-certificates, python-docutils and vala specifically (although its
>possible that we can drop the vala dependency).

... all three of those seem like reasonable enough things to have in
oe-core.  Personally I would quite like to see Vala in there.  So, from
that point of view, I don't have any objection to your proposal.

But, that said, I do still think that there is going to be some
inevitable tension between the desire to make Sato useful in itself and
the desire to have a test environment for oe-core which doesn't add too
many extra dependencies.  So in the longer term I continue to feel that
Sato should probably go away into its own layer (or, at least, a layer
that isn't oe-core) and oe-core itself should gain a dedicated test
suite.  Anybody who wanted to go on using Sato to exercise oe-core would
obviously be free to do so even if it was in some other layer.

p.

Re: Replacing Web in Sato with Midori

[Burton, Ross]

On 8 August 2012 11:39, Phil Blundell <philb@gnu.org> wrote:
> So in the longer term I continue to feel that
> Sato should probably go away into its own layer (or, at least, a layer
> that isn't oe-core) and oe-core itself should gain a dedicated test
> suite.

So what would your ideal dedicated test suite for anything visual (X,
input drivers, video decode, display outputs, etc) be?  I acknowledge
that webkit is a pretty huge dependency so was planning on conceding
that by making it optional, but my plans around Shuku as a Sato
replacement mainly involve removing packages, the only addition being
Midori because it's pretty tough these days to deny that a competent
web browser/HTML app engine is becoming an important feature as many
appliances are moving to HTML/JS instead of other languages (kiosks,
STBs, etc).

Ross

Re: Replacing Web in Sato with Midori

[Phil Blundell]

On Wed, 2012-08-08 at 13:07 +0100, Burton, Ross wrote:
> it's pretty tough these days to deny that a competent
> web browser/HTML app engine is becoming an important feature as many
> appliances are moving to HTML/JS instead of other languages (kiosks,
> STBs, etc).

Yes, I agree, and that's one reason that I think having WebKit in
oe-core proper (rather than relegated to part of the testsuite) would
make sense.

If WebKit did go into oe-core then I agree that using some sort of
WebKit-based browser (which might well be Midori) as part of the test
harness would be an entirely reasonable thing to do.

p.

Re: Replacing Web in Sato with Midori

[Burton, Ross]

On 8 August 2012 13:26, Phil Blundell <philb@gnu.org> wrote:
> Yes, I agree, and that's one reason that I think having WebKit in
> oe-core proper (rather than relegated to part of the testsuite) would
> make sense.
>
> If WebKit did go into oe-core then I agree that using some sort of
> WebKit-based browser (which might well be Midori) as part of the test
> harness would be an entirely reasonable thing to do.

WebKit (well, webkit-gtk 1.8.1) is already in oe-core as it's needed
by Web, but it's not built by default in the core-image-sato as Web is
opt-in.

Ross

Re: Replacing Web in Sato with Midori

[Richard Purdie]

On Wed, 2012-08-08 at 11:39 +0100, Phil Blundell wrote:
> On Wed, 2012-08-08 at 09:41 +0100, Burton, Ross wrote:
> > As everyone who's used it can attest, Web (the optional browser in
> > Sato) is pretty rough.  Part of my plans about replacing Sato with a
> > leaner environment involves replacing it with Midori, and if there
> > isn't any disagreements I'll work on a submission to merge Midori into
> > Sato now for everyone who expects the Sato web browser to be useful.
> 
> Replacing Web with Midori in Sato probably is a fine idea from the point
> of view of those folks who want to use Sato per se.  As far as oe-core
> is concerned, the point of having Sato included is apparently for
> testability and it's not entirely obvious that much extra test coverage
> would be gained by merging Midori.
> 
> Indeed, it's not totally clear that having WebKit in meta-sato is really
> justified by the test coverage it brings.  I think WebKit itself might
> be a reasonable candidate for inclusion in oe-core proper, but the
> current situation of having a slightly half-baked recipe in meta-sato is
> not very satisfactory.
> 
> However...
> 
> >This will involve pulling a few projects from meta-oe to oe-core:
> >ca-certificates, python-docutils and vala specifically (although its
> >possible that we can drop the vala dependency).
> 
> ... all three of those seem like reasonable enough things to have in
> oe-core.  Personally I would quite like to see Vala in there.  So, from
> that point of view, I don't have any objection to your proposal.
> 
> But, that said, I do still think that there is going to be some
> inevitable tension between the desire to make Sato useful in itself and
> the desire to have a test environment for oe-core which doesn't add too
> many extra dependencies.  So in the longer term I continue to feel that
> Sato should probably go away into its own layer (or, at least, a layer
> that isn't oe-core) and oe-core itself should gain a dedicated test
> suite.  Anybody who wanted to go on using Sato to exercise oe-core would
> obviously be free to do so even if it was in some other layer.

I think the intent which perhaps isn't being put clearly is that we're
intending to "morph" sato into a kind of test suite of OE-Core (whether
anything is still "sato" in the end remains to be seen). We appreciate
it needs to be minimal yet have good coverage of the various libs/apis.
Some things will just get moved (eds/pimlico) out, some will go into the
core (e.g. webkit) and so on.

If the webkit recipe is half baked, lets fully bake it as I believe in
doing things properly if you do them at all (which is why web-sato needs
to go).

The changes won't happen overnight but I think Ross' proposal for
sorting the browser as a first step is a good one as part of the larger
plan.

Cheers,

Richard

Re: Replacing Web in Sato with Midori

[Mark Hatle]

On 8/8/12 5:39 AM, Phil Blundell wrote:
> On Wed, 2012-08-08 at 09:41 +0100, Burton, Ross wrote:
>> As everyone who's used it can attest, Web (the optional browser in
>> Sato) is pretty rough.  Part of my plans about replacing Sato with a
>> leaner environment involves replacing it with Midori, and if there
>> isn't any disagreements I'll work on a submission to merge Midori into
>> Sato now for everyone who expects the Sato web browser to be useful.
>
> Replacing Web with Midori in Sato probably is a fine idea from the point
> of view of those folks who want to use Sato per se.  As far as oe-core
> is concerned, the point of having Sato included is apparently for
> testability and it's not entirely obvious that much extra test coverage
> would be gained by merging Midori.

We've been noticing that webkit seems to be pretty good for finding compiler 
bugs.  :P

> Indeed, it's not totally clear that having WebKit in meta-sato is really
> justified by the test coverage it brings.  I think WebKit itself might
> be a reasonable candidate for inclusion in oe-core proper, but the
> current situation of having a slightly half-baked recipe in meta-sato is
> not very satisfactory.
>
> However...
>
>> This will involve pulling a few projects from meta-oe to oe-core:
>> ca-certificates, python-docutils and vala specifically (although its
>> possible that we can drop the vala dependency).
>
> ... all three of those seem like reasonable enough things to have in
> oe-core.  Personally I would quite like to see Vala in there.  So, from
> that point of view, I don't have any objection to your proposal.
>
> But, that said, I do still think that there is going to be some
> inevitable tension between the desire to make Sato useful in itself and
> the desire to have a test environment for oe-core which doesn't add too
> many extra dependencies.  So in the longer term I continue to feel that
> Sato should probably go away into its own layer (or, at least, a layer
> that isn't oe-core) and oe-core itself should gain a dedicated test
> suite.  Anybody who wanted to go on using Sato to exercise oe-core would
> obviously be free to do so even if it was in some other layer.

I agree with the above... we want a sato or sato like environment to do coverage 
tests (primarily verify that subsystems are working together, and graphics is 
functional....)

I'm not against replacing the existing item with Midori, but if we do, it should 
be to exercise (or better exercise) the existing items -- or add the new items.

I know webkit is something that I think is valuable in oe-core, as most embedded 
browsers seem to be based on it these days.  The other items seem like though 
would be valuable as well.  And if all of those can be agreed to go in, then 
Midori seems to be reasonable -as a test case-.

--Mark

> p.
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>