[PATCH 0/1] opkg 0.1.8: respect to the arch when choose the alternatives

[PATCH 0/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Robert Yang]

The following changes since commit 6558abda67956e4fab1304b0cdf03c65f18fabd5:

  qemu: disable vnc-jpeg compression (2012-05-24 08:53:57 +0100)

are available in the git repository at:
  git://git.pokylinux.org/poky-contrib robert/ipk_arch
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ipk_arch

Robert Yang (1):
  opkg 0.1.8: respect to the arch when choose the alternatives

 .../opkg/opkg/respect-to-arch.patch                |   47 ++++++++++++++++++++
 meta/recipes-devtools/opkg/opkg_0.1.8.bb           |    1 +
 meta/recipes-devtools/opkg/opkg_svn.bb             |    1 +
 3 files changed, 49 insertions(+), 0 deletions(-)
 create mode 100644 meta/recipes-devtools/opkg/opkg/respect-to-arch.patch

[PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Robert Yang]

There is a bug if we:
1) bitbake core-image-sato-sdk MACHINE=qemux86
2) bitbake core-image-sato with MACHINE=crownbay

Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
image even if there is one in deploy/ipk/core2 and we have set the
core2's priority higher than i586, when the version in deploy/ipk/i586 is
higher. This doesn't work for us, for example, what the crownbay need is
xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.

This is caused by opkg's selecting mechanism, if there are more than one
candidates which have the same pkg name in the candidate list, for
example, the same pkg with different versions, then it will use the last
one which is the highest version in the list, this doesn't work for us,
it should respect to the arch priorities in such a case.

This is for both denzil and master branch.

[YOCTO #2360]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 .../opkg/opkg/respect-to-arch.patch                |   47 ++++++++++++++++++++
 meta/recipes-devtools/opkg/opkg_0.1.8.bb           |    1 +
 meta/recipes-devtools/opkg/opkg_svn.bb             |    1 +
 3 files changed, 49 insertions(+), 0 deletions(-)
 create mode 100644 meta/recipes-devtools/opkg/opkg/respect-to-arch.patch

diff --git a/meta/recipes-devtools/opkg/opkg/respect-to-arch.patch b/meta/recipes-devtools/opkg/opkg/respect-to-arch.patch
new file mode 100644
index 0000000..6f4537f
--- /dev/null
+++ b/meta/recipes-devtools/opkg/opkg/respect-to-arch.patch
@@ -0,0 +1,47 @@
+pkg_hash.c: respect to the arch priorities when good_pkg_by_name
+
+If there are more than one candidates which have the same pkg name in
+the candidate list, for example, the same pkg with different versions,
+then it will use the last one which is the highest version in the list,
+it should respect to the arch priorities in such a case.
+
+Upstream Status: Pending
+
+Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
+---
+ libopkg/pkg_hash.c |   18 +++++++++++++++---
+ 1 files changed, 15 insertions(+), 3 deletions(-)
+
+diff --git a/libopkg/pkg_hash.c b/libopkg/pkg_hash.c
+index a99cf6b..cc048c8 100644
+--- a/libopkg/pkg_hash.c
++++ b/libopkg/pkg_hash.c
+@@ -376,10 +376,22 @@ pkg_hash_fetch_best_installation_candidate(abstract_pkg_t *apkg,
+           if (constraint_fcn(matching, cdata)) {
+              opkg_msg(DEBUG, "Candidate: %s %s.\n",
+ 			     matching->name, matching->version) ;
+-             good_pkg_by_name = matching;
++             /* Respect to the arch priorities when given alternatives */
++             if (good_pkg_by_name) {
++                 if (matching->arch_priority >= good_pkg_by_name->arch_priority) {
++                     good_pkg_by_name = matching;
++                     opkg_msg(DEBUG, "%s %s wins by priority.\n",
++                         matching->name, matching->version) ;
++                 } else
++                     opkg_msg(DEBUG, "%s %s wins by priority.\n",
++                         good_pkg_by_name->name, good_pkg_by_name->version) ;
++             } else
++                 good_pkg_by_name = matching;
+ 	     /* It has been provided by hand, so it is what user want */
+-             if (matching->provided_by_hand == 1)
+-                break;
++             if (matching->provided_by_hand == 1) {
++                 good_pkg_by_name = matching;
++                 break;
++             }
+           }
+      }
+ 
+-- 
+1.7.1
+
diff --git a/meta/recipes-devtools/opkg/opkg_0.1.8.bb b/meta/recipes-devtools/opkg/opkg_0.1.8.bb
index c206b37..5161824 100644
--- a/meta/recipes-devtools/opkg/opkg_0.1.8.bb
+++ b/meta/recipes-devtools/opkg/opkg_0.1.8.bb
@@ -3,6 +3,7 @@ require opkg.inc
 SRC_URI = "http://opkg.googlecode.com/files/opkg-${PV}.tar.gz \
            file://add_vercmp.patch \
            file://headerfix.patch \
+           file://respect-to-arch.patch \
           "
 
 PR = "${INC_PR}.0"
diff --git a/meta/recipes-devtools/opkg/opkg_svn.bb b/meta/recipes-devtools/opkg/opkg_svn.bb
index c07d393..c3ed973 100644
--- a/meta/recipes-devtools/opkg/opkg_svn.bb
+++ b/meta/recipes-devtools/opkg/opkg_svn.bb
@@ -6,6 +6,7 @@ SRC_URI = "svn://opkg.googlecode.com/svn;module=trunk;proto=http \
            file://fix_installorder.patch \
            file://offline_postinstall.patch\
            file://track_parents.patch \
+           file://respect-to-arch.patch \
 "
 
 S = "${WORKDIR}/trunk"
-- 
1.7.1

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Koen Kooi]

Op 25 mei 2012, om 12:02 heeft Robert Yang het volgende geschreven:

> There is a bug if we:
> 1) bitbake core-image-sato-sdk MACHINE=qemux86
> 2) bitbake core-image-sato with MACHINE=crownbay
> 
> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
> image even if there is one in deploy/ipk/core2 and we have set the
> core2's priority higher than i586, when the version in deploy/ipk/i586 is
> higher. This doesn't work for us, for example, what the crownbay need is
> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
> 
> This is caused by opkg's selecting mechanism, if there are more than one
> candidates which have the same pkg name in the candidate list, for
> example, the same pkg with different versions, then it will use the last
> one which is the highest version in the list, this doesn't work for us,
> it should respect to the arch priorities in such a case.

This is a serious break with the current opkg behaviour and I don't think it's an improvement. Needing different versions for non machine specific packages indicates a more serious bug elsewhere.

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2289 bytes --]

On Fri, May 25, 2012 at 01:19:55PM +0200, Koen Kooi wrote:
> 
> Op 25 mei 2012, om 12:02 heeft Robert Yang het volgende geschreven:
> 
> > There is a bug if we:
> > 1) bitbake core-image-sato-sdk MACHINE=qemux86
> > 2) bitbake core-image-sato with MACHINE=crownbay
> > 
> > Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
> > image even if there is one in deploy/ipk/core2 and we have set the
> > core2's priority higher than i586, when the version in deploy/ipk/i586 is
> > higher. This doesn't work for us, for example, what the crownbay need is
> > xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
> > 
> > This is caused by opkg's selecting mechanism, if there are more than one
> > candidates which have the same pkg name in the candidate list, for
> > example, the same pkg with different versions, then it will use the last
> > one which is the highest version in the list, this doesn't work for us,
> > it should respect to the arch priorities in such a case.
> 
> This is a serious break with the current opkg behaviour and I don't think it's an improvement. Needing different versions for non machine specific packages indicates a more serious bug elsewhere.

It's not the same use-case as those 2 above, but what I don't like on
current opkg behaviour is that it doesn't "reinstall" the package with
the same version when it gets available in arch with higher priority.

e.g. I have armv7a device which has feed urls for armv4t and armv7a
(armv7a of course with higher priority).

foo-1.0 in both feeds armv4t armv7a 

opkg update && opkg install foo -> foo-1.0_armv7a

distro builder publish foo-1.0-r1 sofar only in armv4t feed

opkg update && opkg upgrade -> foo-1.0_armv7a is upgraded to foo-1.0-r1_armv4t)

distro builder publish foo-1.0-r1 also to armv7a feed

opkg update && opkg upgrade -> nothing, but "upgrading" to foo-1.0-r1_armv7a) would be better


On my distro builder I'm trying to prevent this scenario by rsyncing
feeds only after build for *all* supported machines is completed, but
that's still not really atomic operation. (And later I've also started
to filter feeds which gets available on target image).

Cheers,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Richard Purdie]

On Fri, 2012-05-25 at 13:30 +0200, Martin Jansa wrote:
> On Fri, May 25, 2012 at 01:19:55PM +0200, Koen Kooi wrote:
> > 
> > Op 25 mei 2012, om 12:02 heeft Robert Yang het volgende geschreven:
> > 
> > > There is a bug if we:
> > > 1) bitbake core-image-sato-sdk MACHINE=qemux86
> > > 2) bitbake core-image-sato with MACHINE=crownbay
> > > 
> > > Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
> > > image even if there is one in deploy/ipk/core2 and we have set the
> > > core2's priority higher than i586, when the version in deploy/ipk/i586 is
> > > higher. This doesn't work for us, for example, what the crownbay need is
> > > xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
> > > 
> > > This is caused by opkg's selecting mechanism, if there are more than one
> > > candidates which have the same pkg name in the candidate list, for
> > > example, the same pkg with different versions, then it will use the last
> > > one which is the highest version in the list, this doesn't work for us,
> > > it should respect to the arch priorities in such a case.
> > 
> > This is a serious break with the current opkg behaviour and I don't think it's an improvement. Needing different versions for non machine specific packages indicates a more serious bug elsewhere.
> 
> It's not the same use-case as those 2 above, but what I don't like on
> current opkg behaviour is that it doesn't "reinstall" the package with
> the same version when it gets available in arch with higher priority.
> 
> e.g. I have armv7a device which has feed urls for armv4t and armv7a
> (armv7a of course with higher priority).
> 
> foo-1.0 in both feeds armv4t armv7a 
> 
> opkg update && opkg install foo -> foo-1.0_armv7a
> 
> distro builder publish foo-1.0-r1 sofar only in armv4t feed
> 
> opkg update && opkg upgrade -> foo-1.0_armv7a is upgraded to foo-1.0-r1_armv4t)
> 
> distro builder publish foo-1.0-r1 also to armv7a feed
> 
> opkg update && opkg upgrade -> nothing, but "upgrading" to foo-1.0-r1_armv7a) would be better
> 
> 
> On my distro builder I'm trying to prevent this scenario by rsyncing
> feeds only after build for *all* supported machines is completed, but
> that's still not really atomic operation. (And later I've also started
> to filter feeds which gets available on target image).

There is certainly cause to consider what opkg is doing here and whether
it makes sense. Preferring latest version over machine specific feeds is
not perhaps the behaviour we desire or expect. I agree if something
appears in a more optimised architecture, it should consider updating to
it too...

Has anyone talked to opkg upstream about the problems?

Cheers,

Richard

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Robert Yang]

On 05/25/2012 07:19 PM, Koen Kooi wrote:
>
> Op 25 mei 2012, om 12:02 heeft Robert Yang het volgende geschreven:
>
>> There is a bug if we:
>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
>> 2) bitbake core-image-sato with MACHINE=crownbay
>>
>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
>> image even if there is one in deploy/ipk/core2 and we have set the
>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
>> higher. This doesn't work for us, for example, what the crownbay need is
>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
>>
>> This is caused by opkg's selecting mechanism, if there are more than one
>> candidates which have the same pkg name in the candidate list, for
>> example, the same pkg with different versions, then it will use the last
>> one which is the highest version in the list, this doesn't work for us,
>> it should respect to the arch priorities in such a case.
>
> This is a serious break with the current opkg behaviour and I don't think it's an improvement.
> Needing different versions for non machine specific packages indicates a more serious bug elsewhere.

Hi Koen,

Thanks for your reply, here are more explanations.

I think that this patch only affects it's good_pkg_by_name behaviour,
and I don't think that it would introduce other bugs. Here are its
selecting priorities: (From high to low)

1, The pkg set manually
2, The good_pkg_by_name
3, The held pkg
4, Choose the higher arch one
5, Use the latest one

If of them is matched, then the left ones would be ignored, for example,
if the first one matches, others would be ignored.
	
What this patch affects is only the second one(good_by_pkg_name), and doesn't
affects the other four. What it did in the past: If there are both
core2/pkg-1.0.apk and i586/pkg-1.1.apk match, it would use i586/pkg-1.1.apk
since its version is higher and ignore the arch priorities, but we have set the
arch priorities in opkg.conf clearly:

arch i586 31
arch core2 41

What we need is core2/pkg-1.0.apk since we may set the specific pkg version for
the board (for example, the xorg-server for crown-bay), I just let it respect
to the arch priorities in such a case.

For your concerns about the "non machine specific packages", I think what you
mean are the "arch all", "arch any" and "arch noarch", I don't think it would
affect such pkgs, because if a pkg is "arch all", then it should appear in
any arch specified directory, for example, it should not appear in arch i586 or
arch arm.

Please let me know if you have any concerns.

// Robert

> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Robert Yang]

On 05/25/2012 07:30 PM, Martin Jansa wrote:
> On Fri, May 25, 2012 at 01:19:55PM +0200, Koen Kooi wrote:
>>
>> Op 25 mei 2012, om 12:02 heeft Robert Yang het volgende geschreven:
>>
>>> There is a bug if we:
>>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
>>> 2) bitbake core-image-sato with MACHINE=crownbay
>>>
>>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
>>> image even if there is one in deploy/ipk/core2 and we have set the
>>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
>>> higher. This doesn't work for us, for example, what the crownbay need is
>>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
>>>
>>> This is caused by opkg's selecting mechanism, if there are more than one
>>> candidates which have the same pkg name in the candidate list, for
>>> example, the same pkg with different versions, then it will use the last
>>> one which is the highest version in the list, this doesn't work for us,
>>> it should respect to the arch priorities in such a case.
>>
>> This is a serious break with the current opkg behaviour and I don't think it's an improvement. Needing different versions for non machine specific packages indicates a more serious bug elsewhere.
>
> It's not the same use-case as those 2 above, but what I don't like on

Hi Martin,

They are the same cases:-), I think that this patch has also fixed your problem,
the foo-1.0_armv7a will be kept now.

// Robert

> current opkg behaviour is that it doesn't "reinstall" the package with
> the same version when it gets available in arch with higher priority.
>
> e.g. I have armv7a device which has feed urls for armv4t and armv7a
> (armv7a of course with higher priority).
>
> foo-1.0 in both feeds armv4t armv7a
>
> opkg update&&  opkg install foo ->  foo-1.0_armv7a
>
> distro builder publish foo-1.0-r1 sofar only in armv4t feed
>
> opkg update&&  opkg upgrade ->  foo-1.0_armv7a is upgraded to foo-1.0-r1_armv4t)
>
> distro builder publish foo-1.0-r1 also to armv7a feed
>
> opkg update&&  opkg upgrade ->  nothing, but "upgrading" to foo-1.0-r1_armv7a) would be better
>
>
> On my distro builder I'm trying to prevent this scenario by rsyncing
> feeds only after build for *all* supported machines is completed, but
> that's still not really atomic operation. (And later I've also started
> to filter feeds which gets available on target image).
>
> Cheers,
>
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Robert Yang]

On 05/26/2012 10:47 AM, Robert Yang wrote:
>
>
> On 05/25/2012 07:30 PM, Martin Jansa wrote:
>> On Fri, May 25, 2012 at 01:19:55PM +0200, Koen Kooi wrote:
>>>
>>> Op 25 mei 2012, om 12:02 heeft Robert Yang het volgende geschreven:
>>>
>>>> There is a bug if we:
>>>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
>>>> 2) bitbake core-image-sato with MACHINE=crownbay
>>>>
>>>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
>>>> image even if there is one in deploy/ipk/core2 and we have set the
>>>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
>>>> higher. This doesn't work for us, for example, what the crownbay need is
>>>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
>>>>
>>>> This is caused by opkg's selecting mechanism, if there are more than one
>>>> candidates which have the same pkg name in the candidate list, for
>>>> example, the same pkg with different versions, then it will use the last
>>>> one which is the highest version in the list, this doesn't work for us,
>>>> it should respect to the arch priorities in such a case.
>>>
>>> This is a serious break with the current opkg behaviour and I don't think
>>> it's an improvement. Needing different versions for non machine specific
>>> packages indicates a more serious bug elsewhere.
>>
>> It's not the same use-case as those 2 above, but what I don't like on
>
> Hi Martin,
>
> They are the same cases:-), I think that this patch has also fixed your problem,
> the foo-1.0_armv7a will be kept now.
Sorry for the typo,    ~~~~~~~~~~~~~~~ Here should be "will be upgraded".

// Robert

>
> // Robert
>
>> current opkg behaviour is that it doesn't "reinstall" the package with
>> the same version when it gets available in arch with higher priority.
>>
>> e.g. I have armv7a device which has feed urls for armv4t and armv7a
>> (armv7a of course with higher priority).
>>
>> foo-1.0 in both feeds armv4t armv7a
>>
>> opkg update&& opkg install foo -> foo-1.0_armv7a
>>
>> distro builder publish foo-1.0-r1 sofar only in armv4t feed
>>
>> opkg update&& opkg upgrade -> foo-1.0_armv7a is upgraded to foo-1.0-r1_armv4t)
>>
>> distro builder publish foo-1.0-r1 also to armv7a feed
>>
>> opkg update&& opkg upgrade -> nothing, but "upgrading" to foo-1.0-r1_armv7a)
>> would be better
>>
>>
>> On my distro builder I'm trying to prevent this scenario by rsyncing
>> feeds only after build for *all* supported machines is completed, but
>> that's still not really atomic operation. (And later I've also started
>> to filter feeds which gets available on target image).
>>
>> Cheers,
>>
>>
>>
>>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Robert Yang]

I'm sorry that I forgot to update the PR bump, I've updated it
for both opkg_0.1.8.bb and opkg_svn.bb:

-PR = "${INC_PR}.0"
+PR = "${INC_PR}.1"

and pushed the updated patch to:
   git://git.pokylinux.org/poky-contrib robert/ipk_arch
   http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/ipk_arch

// Robert

On 05/25/2012 06:02 PM, Robert Yang wrote:
> There is a bug if we:
> 1) bitbake core-image-sato-sdk MACHINE=qemux86
> 2) bitbake core-image-sato with MACHINE=crownbay
>
> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
> image even if there is one in deploy/ipk/core2 and we have set the
> core2's priority higher than i586, when the version in deploy/ipk/i586 is
> higher. This doesn't work for us, for example, what the crownbay need is
> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
>
> This is caused by opkg's selecting mechanism, if there are more than one
> candidates which have the same pkg name in the candidate list, for
> example, the same pkg with different versions, then it will use the last
> one which is the highest version in the list, this doesn't work for us,
> it should respect to the arch priorities in such a case.
>
> This is for both denzil and master branch.
>
> [YOCTO #2360]
>
> Signed-off-by: Robert Yang<liezhi.yang@windriver.com>
> ---
>   .../opkg/opkg/respect-to-arch.patch                |   47 ++++++++++++++++++++
>   meta/recipes-devtools/opkg/opkg_0.1.8.bb           |    1 +
>   meta/recipes-devtools/opkg/opkg_svn.bb             |    1 +
>   3 files changed, 49 insertions(+), 0 deletions(-)
>   create mode 100644 meta/recipes-devtools/opkg/opkg/respect-to-arch.patch
>
> diff --git a/meta/recipes-devtools/opkg/opkg/respect-to-arch.patch b/meta/recipes-devtools/opkg/opkg/respect-to-arch.patch
> new file mode 100644
> index 0000000..6f4537f
> --- /dev/null
> +++ b/meta/recipes-devtools/opkg/opkg/respect-to-arch.patch
> @@ -0,0 +1,47 @@
> +pkg_hash.c: respect to the arch priorities when good_pkg_by_name
> +
> +If there are more than one candidates which have the same pkg name in
> +the candidate list, for example, the same pkg with different versions,
> +then it will use the last one which is the highest version in the list,
> +it should respect to the arch priorities in such a case.
> +
> +Upstream Status: Pending
> +
> +Signed-off-by: Robert Yang<liezhi.yang@windriver.com>
> +---
> + libopkg/pkg_hash.c |   18 +++++++++++++++---
> + 1 files changed, 15 insertions(+), 3 deletions(-)
> +
> +diff --git a/libopkg/pkg_hash.c b/libopkg/pkg_hash.c
> +index a99cf6b..cc048c8 100644
> +--- a/libopkg/pkg_hash.c
> ++++ b/libopkg/pkg_hash.c
> +@@ -376,10 +376,22 @@ pkg_hash_fetch_best_installation_candidate(abstract_pkg_t *apkg,
> +           if (constraint_fcn(matching, cdata)) {
> +              opkg_msg(DEBUG, "Candidate: %s %s.\n",
> + 			     matching->name, matching->version) ;
> +-             good_pkg_by_name = matching;
> ++             /* Respect to the arch priorities when given alternatives */
> ++             if (good_pkg_by_name) {
> ++                 if (matching->arch_priority>= good_pkg_by_name->arch_priority) {
> ++                     good_pkg_by_name = matching;
> ++                     opkg_msg(DEBUG, "%s %s wins by priority.\n",
> ++                         matching->name, matching->version) ;
> ++                 } else
> ++                     opkg_msg(DEBUG, "%s %s wins by priority.\n",
> ++                         good_pkg_by_name->name, good_pkg_by_name->version) ;
> ++             } else
> ++                 good_pkg_by_name = matching;
> + 	     /* It has been provided by hand, so it is what user want */
> +-             if (matching->provided_by_hand == 1)
> +-                break;
> ++             if (matching->provided_by_hand == 1) {
> ++                 good_pkg_by_name = matching;
> ++                 break;
> ++             }
> +           }
> +      }
> +
> +--
> +1.7.1
> +
> diff --git a/meta/recipes-devtools/opkg/opkg_0.1.8.bb b/meta/recipes-devtools/opkg/opkg_0.1.8.bb
> index c206b37..5161824 100644
> --- a/meta/recipes-devtools/opkg/opkg_0.1.8.bb
> +++ b/meta/recipes-devtools/opkg/opkg_0.1.8.bb
> @@ -3,6 +3,7 @@ require opkg.inc
>   SRC_URI = "http://opkg.googlecode.com/files/opkg-${PV}.tar.gz \
>              file://add_vercmp.patch \
>              file://headerfix.patch \
> +           file://respect-to-arch.patch \
>             "
>
>   PR = "${INC_PR}.0"
> diff --git a/meta/recipes-devtools/opkg/opkg_svn.bb b/meta/recipes-devtools/opkg/opkg_svn.bb
> index c07d393..c3ed973 100644
> --- a/meta/recipes-devtools/opkg/opkg_svn.bb
> +++ b/meta/recipes-devtools/opkg/opkg_svn.bb
> @@ -6,6 +6,7 @@ SRC_URI = "svn://opkg.googlecode.com/svn;module=trunk;proto=http \
>              file://fix_installorder.patch \
>              file://offline_postinstall.patch\
>              file://track_parents.patch \
> +           file://respect-to-arch.patch \
>   "
>
>   S = "${WORKDIR}/trunk"

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3448 bytes --]

On Sat, May 26, 2012 at 10:47:31AM +0800, Robert Yang wrote:
> 
> 
> On 05/25/2012 07:30 PM, Martin Jansa wrote:
> > On Fri, May 25, 2012 at 01:19:55PM +0200, Koen Kooi wrote:
> >>
> >> Op 25 mei 2012, om 12:02 heeft Robert Yang het volgende geschreven:
> >>
> >>> There is a bug if we:
> >>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
> >>> 2) bitbake core-image-sato with MACHINE=crownbay
> >>>
> >>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
> >>> image even if there is one in deploy/ipk/core2 and we have set the
> >>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
> >>> higher. This doesn't work for us, for example, what the crownbay need is
> >>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
> >>>
> >>> This is caused by opkg's selecting mechanism, if there are more than one
> >>> candidates which have the same pkg name in the candidate list, for
> >>> example, the same pkg with different versions, then it will use the last
> >>> one which is the highest version in the list, this doesn't work for us,
> >>> it should respect to the arch priorities in such a case.
> >>
> >> This is a serious break with the current opkg behaviour and I don't think it's an improvement. Needing different versions for non machine specific packages indicates a more serious bug elsewhere.
> >
> > It's not the same use-case as those 2 above, but what I don't like on
> 
> Hi Martin,
> 
> They are the same cases:-), I think that this patch has also fixed your problem,

No, at least not completely the same.

I would prefer to upgrade foo-1.0-r1_armv4t temporary until
foo-1.0-r1_armv7a gets available in feed and that won't happen with your
patch AFAIK.

with your patch:
If you have bar-1.0 which has to be MACHINE_ARCH and in 2.0 bar
developers find way to detect and use all machine capabilities in
runtime, recipe maintainer will switch to TUNE_ARCH, then 
foo-1.0_nokia900.ipk won't be ever upgraded to foo-2.0_armv7a.ipk 
and that's bad.

Cheers,

> the foo-1.0_armv7a will be kept now.
> 
> // Robert
> 
> > current opkg behaviour is that it doesn't "reinstall" the package with
> > the same version when it gets available in arch with higher priority.
> >
> > e.g. I have armv7a device which has feed urls for armv4t and armv7a
> > (armv7a of course with higher priority).
> >
> > foo-1.0 in both feeds armv4t armv7a
> >
> > opkg update&&  opkg install foo ->  foo-1.0_armv7a
> >
> > distro builder publish foo-1.0-r1 sofar only in armv4t feed
> >
> > opkg update&&  opkg upgrade ->  foo-1.0_armv7a is upgraded to foo-1.0-r1_armv4t)
> >
> > distro builder publish foo-1.0-r1 also to armv7a feed
> >
> > opkg update&&  opkg upgrade ->  nothing, but "upgrading" to foo-1.0-r1_armv7a) would be better
> >
> >
> > On my distro builder I'm trying to prevent this scenario by rsyncing
> > feeds only after build for *all* supported machines is completed, but
> > that's still not really atomic operation. (And later I've also started
> > to filter feeds which gets available on target image).
> >
> > Cheers,
> >
> >
> >
> >
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Koen Kooi]

Op 26 mei 2012, om 08:28 heeft Martin Jansa het volgende geschreven:

> On Sat, May 26, 2012 at 10:47:31AM +0800, Robert Yang wrote:
>> 
>> 
>> On 05/25/2012 07:30 PM, Martin Jansa wrote:
>>> On Fri, May 25, 2012 at 01:19:55PM +0200, Koen Kooi wrote:
>>>> 
>>>> Op 25 mei 2012, om 12:02 heeft Robert Yang het volgende geschreven:
>>>> 
>>>>> There is a bug if we:
>>>>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
>>>>> 2) bitbake core-image-sato with MACHINE=crownbay
>>>>> 
>>>>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
>>>>> image even if there is one in deploy/ipk/core2 and we have set the
>>>>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
>>>>> higher. This doesn't work for us, for example, what the crownbay need is
>>>>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
>>>>> 
>>>>> This is caused by opkg's selecting mechanism, if there are more than one
>>>>> candidates which have the same pkg name in the candidate list, for
>>>>> example, the same pkg with different versions, then it will use the last
>>>>> one which is the highest version in the list, this doesn't work for us,
>>>>> it should respect to the arch priorities in such a case.
>>>> 
>>>> This is a serious break with the current opkg behaviour and I don't think it's an improvement. Needing different versions for non machine specific packages indicates a more serious bug elsewhere.
>>> 
>>> It's not the same use-case as those 2 above, but what I don't like on
>> 
>> Hi Martin,
>> 
>> They are the same cases:-), I think that this patch has also fixed your problem,
> 
> No, at least not completely the same.
> 
> I would prefer to upgrade foo-1.0-r1_armv4t temporary until
> foo-1.0-r1_armv7a gets available in feed and that won't happen with your
> patch AFAIK.
> 
> with your patch:
> If you have bar-1.0 which has to be MACHINE_ARCH and in 2.0 bar
> developers find way to detect and use all machine capabilities in
> runtime, recipe maintainer will switch to TUNE_ARCH, then 
> foo-1.0_nokia900.ipk won't be ever upgraded to foo-2.0_armv7a.ipk 
> and that's bad.

And what's worse, the cited usecase is for (slightly paraphrasing):

xserver-xorg 1.11.2 i586
xserver-xorg 1.9.3 i686

Which indicates there is a different, more serious problem at hand. It seems that someone is trying to encode machine specific tweaks to non-machine specific packages. I'm more interested in solving that problem than in changing opkg/rpm behaviour.

There are a number of things that are just not possible to do when supporting multimachine builds and/or multimachine feeds. For example:

machine dogbeachmountain (i686) needs xf86-video-evilpowervr-closedbinary that only works with Xorg 1.9, but machine cherryblossomhighway (i586) can use xf86-video-intel with any xserver-xorg.
If you want both of these machines to work in multimachine builds and/or feeds, you need to lock down xserver-xorg to 1.9 for i*86. If you don't want to lock it down and imgtec won't give you a better binary drop, too bad, stop doing multimachine.
I'm not saying the above situation is what Robert is trying to solve, but it's a situation meta-ti is currently facing with the new binary drop for SGX support. When you have dealt with SGX blobs everything starts to look like an SGX problem :) 

regards,

Koen

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Robert Yang]

On 05/26/2012 02:28 PM, Martin Jansa wrote:
> On Sat, May 26, 2012 at 10:47:31AM +0800, Robert Yang wrote:
>>
>>
>> On 05/25/2012 07:30 PM, Martin Jansa wrote:
>>> On Fri, May 25, 2012 at 01:19:55PM +0200, Koen Kooi wrote:
>>>>
>>>> Op 25 mei 2012, om 12:02 heeft Robert Yang het volgende geschreven:
>>>>
>>>>> There is a bug if we:
>>>>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
>>>>> 2) bitbake core-image-sato with MACHINE=crownbay
>>>>>
>>>>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
>>>>> image even if there is one in deploy/ipk/core2 and we have set the
>>>>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
>>>>> higher. This doesn't work for us, for example, what the crownbay need is
>>>>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
>>>>>
>>>>> This is caused by opkg's selecting mechanism, if there are more than one
>>>>> candidates which have the same pkg name in the candidate list, for
>>>>> example, the same pkg with different versions, then it will use the last
>>>>> one which is the highest version in the list, this doesn't work for us,
>>>>> it should respect to the arch priorities in such a case.
>>>>
>>>> This is a serious break with the current opkg behaviour and I don't think it's an improvement. Needing different versions for non machine specific packages indicates a more serious bug elsewhere.
>>>
>>> It's not the same use-case as those 2 above, but what I don't like on
>>
>> Hi Martin,
>>
>> They are the same cases:-), I think that this patch has also fixed your problem,
>
> No, at least not completely the same.
>
> I would prefer to upgrade foo-1.0-r1_armv4t temporary until

I think it can upgrade foo-1.0-r1_armv4t temporarily, if ipk/armv7a is removed
from the repository, then it would not appear in the opkg.conf, and the armv7a
should be the lowest priority and an unknown arch, and foo-1.0-r1_armv4t will
win, and it would be installed.

I simulated your environment:(with core2 and i586, core2 has a higher version)

1) The foo-1.0_r1.core2 installed

2) Remove the core2 related lines from core-image-sato-1.0-r0/opkg.conf and 
core-image-sato-1.0-r0/rootfs/etc/opkg/arch.conf

3) opkg update && opkg upgrade (foo-1.0_r1.i586 upgraded)

4) Add the core2 related lines back to core-image-sato-1.0-r0/opkg.conf and 
core-image-sato-1.0-r0/rootfs/etc/opkg/arch.conf

5) opkg update && opkg upgrade (foo-1.0_r1.core2 upgraded)

// Robert

> foo-1.0-r1_armv7a gets available in feed and that won't happen with your
> patch AFAIK.
>
> with your patch:
> If you have bar-1.0 which has to be MACHINE_ARCH and in 2.0 bar
> developers find way to detect and use all machine capabilities in
> runtime, recipe maintainer will switch to TUNE_ARCH, then
> foo-1.0_nokia900.ipk won't be ever upgraded to foo-2.0_armv7a.ipk
> and that's bad.
>
> Cheers,
>
>> the foo-1.0_armv7a will be kept now.
>>
>> // Robert
>>
>>> current opkg behaviour is that it doesn't "reinstall" the package with
>>> the same version when it gets available in arch with higher priority.
>>>
>>> e.g. I have armv7a device which has feed urls for armv4t and armv7a
>>> (armv7a of course with higher priority).
>>>
>>> foo-1.0 in both feeds armv4t armv7a
>>>
>>> opkg update&&   opkg install foo ->   foo-1.0_armv7a
>>>
>>> distro builder publish foo-1.0-r1 sofar only in armv4t feed
>>>
>>> opkg update&&   opkg upgrade ->   foo-1.0_armv7a is upgraded to foo-1.0-r1_armv4t)
>>>
>>> distro builder publish foo-1.0-r1 also to armv7a feed
>>>
>>> opkg update&&   opkg upgrade ->   nothing, but "upgrading" to foo-1.0-r1_armv7a) would be better
>>>
>>>
>>> On my distro builder I'm trying to prevent this scenario by rsyncing
>>> feeds only after build for *all* supported machines is completed, but
>>> that's still not really atomic operation. (And later I've also started
>>> to filter feeds which gets available on target image).
>>>
>>> Cheers,
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Openembedded-core mailing list
>>> Openembedded-core@lists.openembedded.org
>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 4896 bytes --]

On Sat, May 26, 2012 at 04:15:09PM +0800, Robert Yang wrote:
> 
> 
> On 05/26/2012 02:28 PM, Martin Jansa wrote:
> > On Sat, May 26, 2012 at 10:47:31AM +0800, Robert Yang wrote:
> >>
> >>
> >> On 05/25/2012 07:30 PM, Martin Jansa wrote:
> >>> On Fri, May 25, 2012 at 01:19:55PM +0200, Koen Kooi wrote:
> >>>>
> >>>> Op 25 mei 2012, om 12:02 heeft Robert Yang het volgende geschreven:
> >>>>
> >>>>> There is a bug if we:
> >>>>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
> >>>>> 2) bitbake core-image-sato with MACHINE=crownbay
> >>>>>
> >>>>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
> >>>>> image even if there is one in deploy/ipk/core2 and we have set the
> >>>>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
> >>>>> higher. This doesn't work for us, for example, what the crownbay need is
> >>>>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
> >>>>>
> >>>>> This is caused by opkg's selecting mechanism, if there are more than one
> >>>>> candidates which have the same pkg name in the candidate list, for
> >>>>> example, the same pkg with different versions, then it will use the last
> >>>>> one which is the highest version in the list, this doesn't work for us,
> >>>>> it should respect to the arch priorities in such a case.
> >>>>
> >>>> This is a serious break with the current opkg behaviour and I don't think it's an improvement. Needing different versions for non machine specific packages indicates a more serious bug elsewhere.
> >>>
> >>> It's not the same use-case as those 2 above, but what I don't like on
> >>
> >> Hi Martin,
> >>
> >> They are the same cases:-), I think that this patch has also fixed your problem,
> >
> > No, at least not completely the same.
> >
> > I would prefer to upgrade foo-1.0-r1_armv4t temporary until
> 
> I think it can upgrade foo-1.0-r1_armv4t temporarily, if ipk/armv7a is removed
> from the repository, then it would not appear in the opkg.conf, and the armv7a
> should be the lowest priority and an unknown arch, and foo-1.0-r1_armv4t will
> win, and it would be installed.
> 
> I simulated your environment:(with core2 and i586, core2 has a higher version)
> 
> 1) The foo-1.0_r1.core2 installed
> 
> 2) Remove the core2 related lines from core-image-sato-1.0-r0/opkg.conf and 
> core-image-sato-1.0-r0/rootfs/etc/opkg/arch.conf
> 
> 3) opkg update && opkg upgrade (foo-1.0_r1.i586 upgraded)
> 
> 4) Add the core2 related lines back to core-image-sato-1.0-r0/opkg.conf and 
> core-image-sato-1.0-r0/rootfs/etc/opkg/arch.conf
> 
> 5) opkg update && opkg upgrade (foo-1.0_r1.core2 upgraded)

Heh, yes of course you can upgrade it with manual aproach on target, but
try to explain it to end users who don't care about OE/opkg and they
just assume that "opkg update && opkg upgrade" does right job and
upgrades their image to newer consistent set of packages.

Cheers,

> 
> // Robert
> 
> > foo-1.0-r1_armv7a gets available in feed and that won't happen with your
> > patch AFAIK.
> >
> > with your patch:
> > If you have bar-1.0 which has to be MACHINE_ARCH and in 2.0 bar
> > developers find way to detect and use all machine capabilities in
> > runtime, recipe maintainer will switch to TUNE_ARCH, then
> > foo-1.0_nokia900.ipk won't be ever upgraded to foo-2.0_armv7a.ipk
> > and that's bad.
> >
> > Cheers,
> >
> >> the foo-1.0_armv7a will be kept now.
> >>
> >> // Robert
> >>
> >>> current opkg behaviour is that it doesn't "reinstall" the package with
> >>> the same version when it gets available in arch with higher priority.
> >>>
> >>> e.g. I have armv7a device which has feed urls for armv4t and armv7a
> >>> (armv7a of course with higher priority).
> >>>
> >>> foo-1.0 in both feeds armv4t armv7a
> >>>
> >>> opkg update&&   opkg install foo ->   foo-1.0_armv7a
> >>>
> >>> distro builder publish foo-1.0-r1 sofar only in armv4t feed
> >>>
> >>> opkg update&&   opkg upgrade ->   foo-1.0_armv7a is upgraded to foo-1.0-r1_armv4t)
> >>>
> >>> distro builder publish foo-1.0-r1 also to armv7a feed
> >>>
> >>> opkg update&&   opkg upgrade ->   nothing, but "upgrading" to foo-1.0-r1_armv7a) would be better
> >>>
> >>>
> >>> On my distro builder I'm trying to prevent this scenario by rsyncing
> >>> feeds only after build for *all* supported machines is completed, but
> >>> that's still not really atomic operation. (And later I've also started
> >>> to filter feeds which gets available on target image).
> >>>
> >>> Cheers,
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Openembedded-core mailing list
> >>> Openembedded-core@lists.openembedded.org
> >>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
> >

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Robert Yang]

On 05/26/2012 04:19 PM, Martin Jansa wrote:
> On Sat, May 26, 2012 at 04:15:09PM +0800, Robert Yang wrote:
>>
>>
>> On 05/26/2012 02:28 PM, Martin Jansa wrote:
>>> On Sat, May 26, 2012 at 10:47:31AM +0800, Robert Yang wrote:
>>>>
>>>>
>>>> On 05/25/2012 07:30 PM, Martin Jansa wrote:
>>>>> On Fri, May 25, 2012 at 01:19:55PM +0200, Koen Kooi wrote:
>>>>>>
>>>>>> Op 25 mei 2012, om 12:02 heeft Robert Yang het volgende geschreven:
>>>>>>
>>>>>>> There is a bug if we:
>>>>>>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
>>>>>>> 2) bitbake core-image-sato with MACHINE=crownbay
>>>>>>>
>>>>>>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
>>>>>>> image even if there is one in deploy/ipk/core2 and we have set the
>>>>>>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
>>>>>>> higher. This doesn't work for us, for example, what the crownbay need is
>>>>>>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
>>>>>>>
>>>>>>> This is caused by opkg's selecting mechanism, if there are more than one
>>>>>>> candidates which have the same pkg name in the candidate list, for
>>>>>>> example, the same pkg with different versions, then it will use the last
>>>>>>> one which is the highest version in the list, this doesn't work for us,
>>>>>>> it should respect to the arch priorities in such a case.
>>>>>>
>>>>>> This is a serious break with the current opkg behaviour and I don't think it's an improvement. Needing different versions for non machine specific packages indicates a more serious bug elsewhere.
>>>>>
>>>>> It's not the same use-case as those 2 above, but what I don't like on
>>>>
>>>> Hi Martin,
>>>>
>>>> They are the same cases:-), I think that this patch has also fixed your problem,
>>>
>>> No, at least not completely the same.
>>>
>>> I would prefer to upgrade foo-1.0-r1_armv4t temporary until
>>
>> I think it can upgrade foo-1.0-r1_armv4t temporarily, if ipk/armv7a is removed
>> from the repository, then it would not appear in the opkg.conf, and the armv7a
>> should be the lowest priority and an unknown arch, and foo-1.0-r1_armv4t will
>> win, and it would be installed.
>>
>> I simulated your environment:(with core2 and i586, core2 has a higher version)
>>
>> 1) The foo-1.0_r1.core2 installed
>>
>> 2) Remove the core2 related lines from core-image-sato-1.0-r0/opkg.conf and
>> core-image-sato-1.0-r0/rootfs/etc/opkg/arch.conf
>>
>> 3) opkg update&&  opkg upgrade (foo-1.0_r1.i586 upgraded)
>>
>> 4) Add the core2 related lines back to core-image-sato-1.0-r0/opkg.conf and
>> core-image-sato-1.0-r0/rootfs/etc/opkg/arch.conf
>>
>> 5) opkg update&&  opkg upgrade (foo-1.0_r1.core2 upgraded)
>
> Heh, yes of course you can upgrade it with manual aproach on target, but
> try to explain it to end users who don't care about OE/opkg and they
> just assume that "opkg update&&  opkg upgrade" does right job and

I'm sorry I don't know what's your real product environment, but the
only "manual" approach that I use is edit the opkg.conf file:-), how can
you remove or add the armv7a feed back if you don't edit the opkg.conf file?
If you remove the ipk files on the disk without edit the opkg.conf file,
there would be errors when you run "opkg update"

// Robert

> upgrades their image to newer consistent set of packages.
>
> Cheers,
>
>>
>> // Robert
>>
>>> foo-1.0-r1_armv7a gets available in feed and that won't happen with your
>>> patch AFAIK.
>>>
>>> with your patch:
>>> If you have bar-1.0 which has to be MACHINE_ARCH and in 2.0 bar
>>> developers find way to detect and use all machine capabilities in
>>> runtime, recipe maintainer will switch to TUNE_ARCH, then
>>> foo-1.0_nokia900.ipk won't be ever upgraded to foo-2.0_armv7a.ipk
>>> and that's bad.
>>>
>>> Cheers,
>>>
>>>> the foo-1.0_armv7a will be kept now.
>>>>
>>>> // Robert
>>>>
>>>>> current opkg behaviour is that it doesn't "reinstall" the package with
>>>>> the same version when it gets available in arch with higher priority.
>>>>>
>>>>> e.g. I have armv7a device which has feed urls for armv4t and armv7a
>>>>> (armv7a of course with higher priority).
>>>>>
>>>>> foo-1.0 in both feeds armv4t armv7a
>>>>>
>>>>> opkg update&&    opkg install foo ->    foo-1.0_armv7a
>>>>>
>>>>> distro builder publish foo-1.0-r1 sofar only in armv4t feed
>>>>>
>>>>> opkg update&&    opkg upgrade ->    foo-1.0_armv7a is upgraded to foo-1.0-r1_armv4t)
>>>>>
>>>>> distro builder publish foo-1.0-r1 also to armv7a feed
>>>>>
>>>>> opkg update&&    opkg upgrade ->    nothing, but "upgrading" to foo-1.0-r1_armv7a) would be better
>>>>>
>>>>>
>>>>> On my distro builder I'm trying to prevent this scenario by rsyncing
>>>>> feeds only after build for *all* supported machines is completed, but
>>>>> that's still not really atomic operation. (And later I've also started
>>>>> to filter feeds which gets available on target image).
>>>>>
>>>>> Cheers,
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Openembedded-core mailing list
>>>>> Openembedded-core@lists.openembedded.org
>>>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>>
>

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 6089 bytes --]

On Sat, May 26, 2012 at 04:35:32PM +0800, Robert Yang wrote:
> 
> 
> On 05/26/2012 04:19 PM, Martin Jansa wrote:
> > On Sat, May 26, 2012 at 04:15:09PM +0800, Robert Yang wrote:
> >>
> >>
> >> On 05/26/2012 02:28 PM, Martin Jansa wrote:
> >>> On Sat, May 26, 2012 at 10:47:31AM +0800, Robert Yang wrote:
> >>>>
> >>>>
> >>>> On 05/25/2012 07:30 PM, Martin Jansa wrote:
> >>>>> On Fri, May 25, 2012 at 01:19:55PM +0200, Koen Kooi wrote:
> >>>>>>
> >>>>>> Op 25 mei 2012, om 12:02 heeft Robert Yang het volgende geschreven:
> >>>>>>
> >>>>>>> There is a bug if we:
> >>>>>>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
> >>>>>>> 2) bitbake core-image-sato with MACHINE=crownbay
> >>>>>>>
> >>>>>>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
> >>>>>>> image even if there is one in deploy/ipk/core2 and we have set the
> >>>>>>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
> >>>>>>> higher. This doesn't work for us, for example, what the crownbay need is
> >>>>>>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
> >>>>>>>
> >>>>>>> This is caused by opkg's selecting mechanism, if there are more than one
> >>>>>>> candidates which have the same pkg name in the candidate list, for
> >>>>>>> example, the same pkg with different versions, then it will use the last
> >>>>>>> one which is the highest version in the list, this doesn't work for us,
> >>>>>>> it should respect to the arch priorities in such a case.
> >>>>>>
> >>>>>> This is a serious break with the current opkg behaviour and I don't think it's an improvement. Needing different versions for non machine specific packages indicates a more serious bug elsewhere.
> >>>>>
> >>>>> It's not the same use-case as those 2 above, but what I don't like on
> >>>>
> >>>> Hi Martin,
> >>>>
> >>>> They are the same cases:-), I think that this patch has also fixed your problem,
> >>>
> >>> No, at least not completely the same.
> >>>
> >>> I would prefer to upgrade foo-1.0-r1_armv4t temporary until
> >>
> >> I think it can upgrade foo-1.0-r1_armv4t temporarily, if ipk/armv7a is removed
> >> from the repository, then it would not appear in the opkg.conf, and the armv7a
> >> should be the lowest priority and an unknown arch, and foo-1.0-r1_armv4t will
> >> win, and it would be installed.
> >>
> >> I simulated your environment:(with core2 and i586, core2 has a higher version)
> >>
> >> 1) The foo-1.0_r1.core2 installed
> >>
> >> 2) Remove the core2 related lines from core-image-sato-1.0-r0/opkg.conf and
> >> core-image-sato-1.0-r0/rootfs/etc/opkg/arch.conf
> >>
> >> 3) opkg update&&  opkg upgrade (foo-1.0_r1.i586 upgraded)
> >>
> >> 4) Add the core2 related lines back to core-image-sato-1.0-r0/opkg.conf and
> >> core-image-sato-1.0-r0/rootfs/etc/opkg/arch.conf
> >>
> >> 5) opkg update&&  opkg upgrade (foo-1.0_r1.core2 upgraded)
> >
> > Heh, yes of course you can upgrade it with manual aproach on target, but
> > try to explain it to end users who don't care about OE/opkg and they
> > just assume that "opkg update&&  opkg upgrade" does right job and
> 
> I'm sorry I don't know what's your real product environment, but the
> only "manual" approach that I use is edit the opkg.conf file:-), how can
> you remove or add the armv7a feed back if you don't edit the opkg.conf file?
> If you remove the ipk files on the disk without edit the opkg.conf file,
> there would be errors when you run "opkg update"

I don't want to remove and add any feed on any target, my whole use case
is that distro feeds for multiple machines are not populated atomicaly
and if someone is unlucky (and does "opkg update && opkg upgrade" in wrong time)
he will get stuck with foo-1.0-r1_armv4t even when few secs later foo-1.0-r1_armv7a
gets available few sec after that (and next "opkg update && opkg
upgrade" wont help him)

Cheers,

> 
> // Robert
> 
> > upgrades their image to newer consistent set of packages.
> >
> > Cheers,
> >
> >>
> >> // Robert
> >>
> >>> foo-1.0-r1_armv7a gets available in feed and that won't happen with your
> >>> patch AFAIK.
> >>>
> >>> with your patch:
> >>> If you have bar-1.0 which has to be MACHINE_ARCH and in 2.0 bar
> >>> developers find way to detect and use all machine capabilities in
> >>> runtime, recipe maintainer will switch to TUNE_ARCH, then
> >>> foo-1.0_nokia900.ipk won't be ever upgraded to foo-2.0_armv7a.ipk
> >>> and that's bad.
> >>>
> >>> Cheers,
> >>>
> >>>> the foo-1.0_armv7a will be kept now.
> >>>>
> >>>> // Robert
> >>>>
> >>>>> current opkg behaviour is that it doesn't "reinstall" the package with
> >>>>> the same version when it gets available in arch with higher priority.
> >>>>>
> >>>>> e.g. I have armv7a device which has feed urls for armv4t and armv7a
> >>>>> (armv7a of course with higher priority).
> >>>>>
> >>>>> foo-1.0 in both feeds armv4t armv7a
> >>>>>
> >>>>> opkg update&&    opkg install foo ->    foo-1.0_armv7a
> >>>>>
> >>>>> distro builder publish foo-1.0-r1 sofar only in armv4t feed
> >>>>>
> >>>>> opkg update&&    opkg upgrade ->    foo-1.0_armv7a is upgraded to foo-1.0-r1_armv4t)
> >>>>>
> >>>>> distro builder publish foo-1.0-r1 also to armv7a feed
> >>>>>
> >>>>> opkg update&&    opkg upgrade ->    nothing, but "upgrading" to foo-1.0-r1_armv7a) would be better
> >>>>>
> >>>>>
> >>>>> On my distro builder I'm trying to prevent this scenario by rsyncing
> >>>>> feeds only after build for *all* supported machines is completed, but
> >>>>> that's still not really atomic operation. (And later I've also started
> >>>>> to filter feeds which gets available on target image).
> >>>>>
> >>>>> Cheers,
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Openembedded-core mailing list
> >>>>> Openembedded-core@lists.openembedded.org
> >>>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
> >>>
> >

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Robert Yang]

On 05/26/2012 04:07 PM, Koen Kooi wrote:
>
> Op 26 mei 2012, om 08:28 heeft Martin Jansa het volgende geschreven:
>
>> On Sat, May 26, 2012 at 10:47:31AM +0800, Robert Yang wrote:
>>>
>>>
>>> On 05/25/2012 07:30 PM, Martin Jansa wrote:
>>>> On Fri, May 25, 2012 at 01:19:55PM +0200, Koen Kooi wrote:
>>>>>
>>>>> Op 25 mei 2012, om 12:02 heeft Robert Yang het volgende geschreven:
>>>>>
>>>>>> There is a bug if we:
>>>>>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
>>>>>> 2) bitbake core-image-sato with MACHINE=crownbay
>>>>>>
>>>>>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
>>>>>> image even if there is one in deploy/ipk/core2 and we have set the
>>>>>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
>>>>>> higher. This doesn't work for us, for example, what the crownbay need is
>>>>>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
>>>>>>
>>>>>> This is caused by opkg's selecting mechanism, if there are more than one
>>>>>> candidates which have the same pkg name in the candidate list, for
>>>>>> example, the same pkg with different versions, then it will use the last
>>>>>> one which is the highest version in the list, this doesn't work for us,
>>>>>> it should respect to the arch priorities in such a case.
>>>>>
>>>>> This is a serious break with the current opkg behaviour and I don't think it's an improvement. Needing different versions for non machine specific packages indicates a more serious bug elsewhere.
>>>>
>>>> It's not the same use-case as those 2 above, but what I don't like on
>>>
>>> Hi Martin,
>>>
>>> They are the same cases:-), I think that this patch has also fixed your problem,
>>
>> No, at least not completely the same.
>>
>> I would prefer to upgrade foo-1.0-r1_armv4t temporary until
>> foo-1.0-r1_armv7a gets available in feed and that won't happen with your
>> patch AFAIK.
>>
>> with your patch:
>> If you have bar-1.0 which has to be MACHINE_ARCH and in 2.0 bar
>> developers find way to detect and use all machine capabilities in
>> runtime, recipe maintainer will switch to TUNE_ARCH, then
>> foo-1.0_nokia900.ipk won't be ever upgraded to foo-2.0_armv7a.ipk
>> and that's bad.
>
> And what's worse, the cited usecase is for (slightly paraphrasing):
>
> xserver-xorg 1.11.2 i586
> xserver-xorg 1.9.3 i686
>
> Which indicates there is a different, more serious problem at hand. It seems that someone is trying to encode machine specific tweaks to non-machine specific packages. I'm more interested in solving that problem than in changing opkg/rpm behaviour.
>

Yes, fixing the pkg itself is the first way that I had thought:-), but as you
said below, the crownbay board must work with xserver-xorg 1.9.3, and we
can't remove the i586 from crownbay's ARCHs, so the last way I left is fixing
opkg. I think that respect to the arch priority in the multimachine builds
is reasonable as had you suggested before.

// Robert

> There are a number of things that are just not possible to do when supporting multimachine builds and/or multimachine feeds. For example:
>
> machine dogbeachmountain (i686) needs xf86-video-evilpowervr-closedbinary that only works with Xorg 1.9, but machine cherryblossomhighway (i586) can use xf86-video-intel with any xserver-xorg.
> If you want both of these machines to work in multimachine builds and/or feeds, you need to lock down xserver-xorg to 1.9 for i*86. If you don't want to lock it down and imgtec won't give you a better binary drop, too bad, stop doing multimachine.
> I'm not saying the above situation is what Robert is trying to solve, but it's a situation meta-ti is currently facing with the new binary drop for SGX support. When you have dealt with SGX blobs everything starts to look like an SGX problem :)
>
> regards,
>
> Koen
>

[PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Robert Yang]

There is a bug if we:
1) bitbake core-image-sato-sdk MACHINE=qemux86
2) bitbake core-image-sato with MACHINE=crownbay

Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
image even if there is one in deploy/ipk/core2 and we have set the
core2's priority higher than i586, when the version in deploy/ipk/i586 is
higher. This doesn't work for us, for example, what the crownbay need is
xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.

This is caused by opkg's selecting mechanism, if there are more than one
candidates which have the same pkg name in the candidate list, for
example, the same pkg with different versions, then it will use the last
one which is the highest version in the list, this doesn't work for us,
it should respect to the arch priorities in such a case.

This is for both denzil and master branch.

[YOCTO #2360]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 .../opkg/opkg/respect-to-arch.patch                |   47 ++++++++++++++++++++
 meta/recipes-devtools/opkg/opkg_0.1.8.bb           |    3 +-
 meta/recipes-devtools/opkg/opkg_svn.bb             |    3 +-
 3 files changed, 51 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-devtools/opkg/opkg/respect-to-arch.patch

diff --git a/meta/recipes-devtools/opkg/opkg/respect-to-arch.patch b/meta/recipes-devtools/opkg/opkg/respect-to-arch.patch
new file mode 100644
index 0000000..6f4537f
--- /dev/null
+++ b/meta/recipes-devtools/opkg/opkg/respect-to-arch.patch
@@ -0,0 +1,47 @@
+pkg_hash.c: respect to the arch priorities when good_pkg_by_name
+
+If there are more than one candidates which have the same pkg name in
+the candidate list, for example, the same pkg with different versions,
+then it will use the last one which is the highest version in the list,
+it should respect to the arch priorities in such a case.
+
+Upstream Status: Pending
+
+Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
+---
+ libopkg/pkg_hash.c |   18 +++++++++++++++---
+ 1 files changed, 15 insertions(+), 3 deletions(-)
+
+diff --git a/libopkg/pkg_hash.c b/libopkg/pkg_hash.c
+index a99cf6b..cc048c8 100644
+--- a/libopkg/pkg_hash.c
++++ b/libopkg/pkg_hash.c
+@@ -376,10 +376,22 @@ pkg_hash_fetch_best_installation_candidate(abstract_pkg_t *apkg,
+           if (constraint_fcn(matching, cdata)) {
+              opkg_msg(DEBUG, "Candidate: %s %s.\n",
+ 			     matching->name, matching->version) ;
+-             good_pkg_by_name = matching;
++             /* Respect to the arch priorities when given alternatives */
++             if (good_pkg_by_name) {
++                 if (matching->arch_priority >= good_pkg_by_name->arch_priority) {
++                     good_pkg_by_name = matching;
++                     opkg_msg(DEBUG, "%s %s wins by priority.\n",
++                         matching->name, matching->version) ;
++                 } else
++                     opkg_msg(DEBUG, "%s %s wins by priority.\n",
++                         good_pkg_by_name->name, good_pkg_by_name->version) ;
++             } else
++                 good_pkg_by_name = matching;
+ 	     /* It has been provided by hand, so it is what user want */
+-             if (matching->provided_by_hand == 1)
+-                break;
++             if (matching->provided_by_hand == 1) {
++                 good_pkg_by_name = matching;
++                 break;
++             }
+           }
+      }
+ 
+-- 
+1.7.1
+
diff --git a/meta/recipes-devtools/opkg/opkg_0.1.8.bb b/meta/recipes-devtools/opkg/opkg_0.1.8.bb
index c206b37..09ff3b5 100644
--- a/meta/recipes-devtools/opkg/opkg_0.1.8.bb
+++ b/meta/recipes-devtools/opkg/opkg_0.1.8.bb
@@ -3,6 +3,7 @@ require opkg.inc
 SRC_URI = "http://opkg.googlecode.com/files/opkg-${PV}.tar.gz \
            file://add_vercmp.patch \
            file://headerfix.patch \
+           file://respect-to-arch.patch \
           "
 
-PR = "${INC_PR}.0"
+PR = "${INC_PR}.1"
diff --git a/meta/recipes-devtools/opkg/opkg_svn.bb b/meta/recipes-devtools/opkg/opkg_svn.bb
index c07d393..890ed52 100644
--- a/meta/recipes-devtools/opkg/opkg_svn.bb
+++ b/meta/recipes-devtools/opkg/opkg_svn.bb
@@ -6,6 +6,7 @@ SRC_URI = "svn://opkg.googlecode.com/svn;module=trunk;proto=http \
            file://fix_installorder.patch \
            file://offline_postinstall.patch\
            file://track_parents.patch \
+           file://respect-to-arch.patch \
 "
 
 S = "${WORKDIR}/trunk"
@@ -13,4 +14,4 @@ S = "${WORKDIR}/trunk"
 SRCREV = "633"
 PV = "0.1.8+svnr${SRCPV}"
 
-PR = "${INC_PR}.0"
+PR = "${INC_PR}.1"
-- 
1.7.1

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Koen Kooi]

Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:

> There is a bug if we:
> 1) bitbake core-image-sato-sdk MACHINE=qemux86
> 2) bitbake core-image-sato with MACHINE=crownbay
> 
> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
> image even if there is one in deploy/ipk/core2 and we have set the
> core2's priority higher than i586, when the version in deploy/ipk/i586 is
> higher. This doesn't work for us, for example, what the crownbay need is
> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.

And this is working exactly as intended. Don't break opkg because your hardware driver situation sucks.

So: NAK on this patch.

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Robert Yang]

On 05/31/2012 11:01 PM, Koen Kooi wrote:
>
> Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:
>
>> There is a bug if we:
>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
>> 2) bitbake core-image-sato with MACHINE=crownbay
>>
>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
>> image even if there is one in deploy/ipk/core2 and we have set the
>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
>> higher. This doesn't work for us, for example, what the crownbay need is
>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
>
> And this is working exactly as intended. Don't break opkg because your hardware driver situation sucks.
>

Hi Koen,

Thanks for your great patient on this patch:-), I'm afraid that this is
not working exactly as intended, since the rpm or dpkg doesn't have this
issue. (I have fixed other issue for rpm, but not the same as this one,
the xserver-xorg-1.9.3.rpm had installed correctly before the fix for rpm).
and dpkg works well without any fix.

I'm also glad that if we won't fix it.

// Robert


> So: NAK on this patch.
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Richard Purdie]

On Thu, 2012-05-31 at 17:01 +0200, Koen Kooi wrote:
> Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:
> 
> > There is a bug if we:
> > 1) bitbake core-image-sato-sdk MACHINE=qemux86
> > 2) bitbake core-image-sato with MACHINE=crownbay
> > 
> > Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
> > image even if there is one in deploy/ipk/core2 and we have set the
> > core2's priority higher than i586, when the version in deploy/ipk/i586 is
> > higher. This doesn't work for us, for example, what the crownbay need is
> > xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
> 
> And this is working exactly as intended. Don't break opkg because your
> hardware driver situation sucks.
> 
> So: NAK on this patch.

I think we do have a problem here. For example, the system is ignoring a
PREFERRED_VERSION directive here by building one thing and then
installing another. We're also inconsistent between the dpkg/rpm and
opkg backends. There is therefore definitely some kind of user
experience issue at stake here since this behaviour is not obvious,
expected or particularly correct.

The fact the example is hardware related is not particularly relevant,
its the bigger picture I worry about. I know that hardware issue sucks
and many people on this list have experienced pain due to it, we'd all
like it to go away. Using this as a reason not to examine and
potentially fix some problematic package manager behaviour is not right
IMO though. The world isn't perfect, sucky hardware/software exists, we
need to work with it.

So to be honest I'm leaning towards taking the patch. I would be
interested in other people's opinions though...

Cheers,

Richard

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Koen Kooi]

Op 1 jun. 2012, om 10:17 heeft Richard Purdie het volgende geschreven:

> On Thu, 2012-05-31 at 17:01 +0200, Koen Kooi wrote:
>> Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:
>> 
>>> There is a bug if we:
>>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
>>> 2) bitbake core-image-sato with MACHINE=crownbay
>>> 
>>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
>>> image even if there is one in deploy/ipk/core2 and we have set the
>>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
>>> higher. This doesn't work for us, for example, what the crownbay need is
>>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
>> 
>> And this is working exactly as intended. Don't break opkg because your
>> hardware driver situation sucks.
>> 
>> So: NAK on this patch.
> 
> I think we do have a problem here. For example, the system is ignoring a
> PREFERRED_VERSION directive here

A PREFERRED_VERSION set in a machine.conf, which is the wrong place. Let's change the above build sequence to this:

1) MACHINE=qemux86 bitbake xserver-xorg
2) MACHINE=othercore2machine bitbake xserver-xorg
3) MACHINE=crownbay bitbake xserver-xorg

Oh look, I get 1.11.2 on crownbay regardless of this patch.

> by building one thing and then
> installing another. We're also inconsistent between the dpkg/rpm and
> opkg backends. There is therefore definitely some kind of user
> experience issue at stake here since this behaviour is not obvious,
> expected or particularly correct.
> 
> The fact the example is hardware related is not particularly relevant,
> its the bigger picture I worry about

I also worry about the bigger picture, especially about what Martin said about this breaking feeds.

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Richard Purdie]

On Fri, 2012-06-01 at 11:04 +0200, Koen Kooi wrote:
> Op 1 jun. 2012, om 10:17 heeft Richard Purdie het volgende geschreven:
> 
> > On Thu, 2012-05-31 at 17:01 +0200, Koen Kooi wrote:
> >> Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:
> >> 
> >>> There is a bug if we:
> >>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
> >>> 2) bitbake core-image-sato with MACHINE=crownbay
> >>> 
> >>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
> >>> image even if there is one in deploy/ipk/core2 and we have set the
> >>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
> >>> higher. This doesn't work for us, for example, what the crownbay need is
> >>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
> >> 
> >> And this is working exactly as intended. Don't break opkg because your
> >> hardware driver situation sucks.
> >> 
> >> So: NAK on this patch.
> > 
> > I think we do have a problem here. For example, the system is ignoring a
> > PREFERRED_VERSION directive here
> 
> A PREFERRED_VERSION set in a machine.conf, which is the wrong place.

Its strongly not recommended. You can do it and if things are setup
correctly, we do support machine specific alterations however...

>  Let's change the above build sequence to this:
> 
> 1) MACHINE=qemux86 bitbake xserver-xorg
> 2) MACHINE=othercore2machine bitbake xserver-xorg
> 3) MACHINE=crownbay bitbake xserver-xorg
> 
> Oh look, I get 1.11.2 on crownbay regardless of this patch.

I've been assuming this xserver is marked as machine specific. If its
not, that is a bug and we can fix that.

> > by building one thing and then
> > installing another. We're also inconsistent between the dpkg/rpm and
> > opkg backends. There is therefore definitely some kind of user
> > experience issue at stake here since this behaviour is not obvious,
> > expected or particularly correct.
> > 
> > The fact the example is hardware related is not particularly relevant,
> > its the bigger picture I worry about
> 
> I also worry about the bigger picture, especially about what Martin
> said about this breaking feeds.

As far as I understood Martin's comments, this actually helps avoid some
of the issues he's been experiencing with feeds?

Martin has a problem where machines are ending up with unoptimised
versions of code on them and it would be good to fix opkg behaviour to
do what people expect...

Cheers,

Richard

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Koen Kooi]

Op 1 jun. 2012, om 12:02 heeft Richard Purdie het volgende geschreven:

> On Fri, 2012-06-01 at 11:04 +0200, Koen Kooi wrote:
>> Op 1 jun. 2012, om 10:17 heeft Richard Purdie het volgende geschreven:
>> 
>>> On Thu, 2012-05-31 at 17:01 +0200, Koen Kooi wrote:
>>>> Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:
>>>> 
>>>>> There is a bug if we:
>>>>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
>>>>> 2) bitbake core-image-sato with MACHINE=crownbay
>>>>> 
>>>>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
>>>>> image even if there is one in deploy/ipk/core2 and we have set the
>>>>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
>>>>> higher. This doesn't work for us, for example, what the crownbay need is
>>>>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
>>>> 
>>>> And this is working exactly as intended. Don't break opkg because your
>>>> hardware driver situation sucks.
>>>> 
>>>> So: NAK on this patch.
>>> 
>>> I think we do have a problem here. For example, the system is ignoring a
>>> PREFERRED_VERSION directive here
>> 
>> A PREFERRED_VERSION set in a machine.conf, which is the wrong place.
> 
> Its strongly not recommended. You can do it and if things are setup
> correctly, we do support machine specific alterations however...
> 
>> Let's change the above build sequence to this:
>> 
>> 1) MACHINE=qemux86 bitbake xserver-xorg
>> 2) MACHINE=othercore2machine bitbake xserver-xorg
>> 3) MACHINE=crownbay bitbake xserver-xorg
>> 
>> Oh look, I get 1.11.2 on crownbay regardless of this patch.
> 
> I've been assuming this xserver is marked as machine specific. If its
> not, that is a bug and we can fix that.

the X server not machine specific and that is NOT a bug. The recipe for the DDX only works with a specific X version. The real problem here is that xf86-video-something (again, not machine specific, think pci cards) has a strict requirement on the x server version. This is now purely a DISTRO problem. A few possible solutions are:

1) Lock X to 1.9.3 for all compatible x86 archs in your DISTRO
2) Lock X to 1.11.2 for all compatible x86 archs in your DISTRO, live with the crownbay breakage
3) Only use a package manager with no notion of compatible archs (dpkg) or with strong version pinning (rpm)
4) Be really careful with the build sequence and structure your feed configs to not have compatible archs in their feed URIs
5) remove 'x11' from DISTRO_FEATURES

This is a point where you cannot make everybody happy. But as you say below, let's decouple the above problem from the arch vs version discussion.

>>> by building one thing and then
>>> installing another. We're also inconsistent between the dpkg/rpm and
>>> opkg backends. There is therefore definitely some kind of user
>>> experience issue at stake here since this behaviour is not obvious,
>>> expected or particularly correct.
>>> 
>>> The fact the example is hardware related is not particularly relevant,
>>> its the bigger picture I worry about
>> 
>> I also worry about the bigger picture, especially about what Martin
>> said about this breaking feeds.
> 
> As far as I understood Martin's comments, this actually helps avoid some
> of the issues he's been experiencing with feeds?

No, it allows you to add a package-of-death. Example:

broken-app_1.0_machine.ipk is in the feeds, being machine specific
broken-apps author fixes the machine specific bits properly
broken-app_2.0_i586.ipk hits the feeds and is not picked up.

> Martin has a problem where machines are ending up with unoptimised
> versions of code on them and it would be good to fix opkg behaviour to
> do what people expect...

Yes, feed updates are not atomic, but this patch breaks moving packages between archs like the broken-app example above. In angstrom a machine doesn't see feeds with compatible archs at runtime, e.g. beagleboard sees beagleboard, armv7a and noarch URIs. This reduces these kind of problems to images built with OE. I must note that this setup for feed URIs was done out of bandwidth, storage and ram concerns (good old ipkg), not to avoid Martins problems :)

regards,

Koen

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Robert Yang]

On 06/01/2012 06:35 PM, Koen Kooi wrote:
>
> Op 1 jun. 2012, om 12:02 heeft Richard Purdie het volgende geschreven:
>
>> On Fri, 2012-06-01 at 11:04 +0200, Koen Kooi wrote:
>>> Op 1 jun. 2012, om 10:17 heeft Richard Purdie het volgende geschreven:
>>>
>>>> On Thu, 2012-05-31 at 17:01 +0200, Koen Kooi wrote:
>>>>> Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:
>>>>>
>>>>>> There is a bug if we:
>>>>>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
>>>>>> 2) bitbake core-image-sato with MACHINE=crownbay
>>>>>>
>>>>>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
>>>>>> image even if there is one in deploy/ipk/core2 and we have set the
>>>>>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
>>>>>> higher. This doesn't work for us, for example, what the crownbay need is
>>>>>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
>>>>>
>>>>> And this is working exactly as intended. Don't break opkg because your
>>>>> hardware driver situation sucks.
>>>>>
>>>>> So: NAK on this patch.
>>>>
>>>> I think we do have a problem here. For example, the system is ignoring a
>>>> PREFERRED_VERSION directive here
>>>
>>> A PREFERRED_VERSION set in a machine.conf, which is the wrong place.
>>
>> Its strongly not recommended. You can do it and if things are setup
>> correctly, we do support machine specific alterations however...
>>
>>> Let's change the above build sequence to this:
>>>
>>> 1) MACHINE=qemux86 bitbake xserver-xorg
>>> 2) MACHINE=othercore2machine bitbake xserver-xorg
>>> 3) MACHINE=crownbay bitbake xserver-xorg
>>>
>>> Oh look, I get 1.11.2 on crownbay regardless of this patch.
>>
>> I've been assuming this xserver is marked as machine specific. If its
>> not, that is a bug and we can fix that.
>
> the X server not machine specific and that is NOT a bug. The recipe for the DDX only works with a specific X version. The real problem here is that xf86-video-something (again, not machine specific, think pci cards) has a strict requirement on the x server version. This is now purely a DISTRO problem. A few possible solutions are:
>
> 1) Lock X to 1.9.3 for all compatible x86 archs in your DISTRO
> 2) Lock X to 1.11.2 for all compatible x86 archs in your DISTRO, live with the crownbay breakage
> 3) Only use a package manager with no notion of compatible archs (dpkg) or with strong version pinning (rpm)
> 4) Be really careful with the build sequence and structure your feed configs to not have compatible archs in their feed URIs
> 5) remove 'x11' from DISTRO_FEATURES
>
> This is a point where you cannot make everybody happy. But as you say below, let's decouple the above problem from the arch vs version discussion.
>
>>>> by building one thing and then
>>>> installing another. We're also inconsistent between the dpkg/rpm and
>>>> opkg backends. There is therefore definitely some kind of user
>>>> experience issue at stake here since this behaviour is not obvious,
>>>> expected or particularly correct.
>>>>
>>>> The fact the example is hardware related is not particularly relevant,
>>>> its the bigger picture I worry about
>>>
>>> I also worry about the bigger picture, especially about what Martin
>>> said about this breaking feeds.
>>
>> As far as I understood Martin's comments, this actually helps avoid some
>> of the issues he's been experiencing with feeds?
>
> No, it allows you to add a package-of-death. Example:
>
> broken-app_1.0_machine.ipk is in the feeds, being machine specific
> broken-apps author fixes the machine specific bits properly
> broken-app_2.0_i586.ipk hits the feeds and is not picked up.

I think that the broken-app_2.0_i586.ipk came unexpectedly,
there was a broken-app_1.0_machine.ipk (maybe also broken-app_1.0_i586.ipk,
but it didn't matter), if it has been fixed to version 2.0, then there should
be also a broken-app_2.0_machine.ipk (and it would be picked up).

// Robert

>
>> Martin has a problem where machines are ending up with unoptimised
>> versions of code on them and it would be good to fix opkg behaviour to
>> do what people expect...
>
> Yes, feed updates are not atomic, but this patch breaks moving packages between archs like the broken-app example above. In angstrom a machine doesn't see feeds with compatible archs at runtime, e.g. beagleboard sees beagleboard, armv7a and noarch URIs. This reduces these kind of problems to images built with OE. I must note that this setup for feed URIs was done out of bandwidth, storage and ram concerns (good old ipkg), not to avoid Martins problems :)
>
> regards,
>
> Koen
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 5874 bytes --]

On Mon, Jun 04, 2012 at 05:31:26PM +0800, Robert Yang wrote:
> 
> 
> On 06/01/2012 06:35 PM, Koen Kooi wrote:
> >
> > Op 1 jun. 2012, om 12:02 heeft Richard Purdie het volgende geschreven:
> >
> >> On Fri, 2012-06-01 at 11:04 +0200, Koen Kooi wrote:
> >>> Op 1 jun. 2012, om 10:17 heeft Richard Purdie het volgende geschreven:
> >>>
> >>>> On Thu, 2012-05-31 at 17:01 +0200, Koen Kooi wrote:
> >>>>> Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:
> >>>>>
> >>>>>> There is a bug if we:
> >>>>>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
> >>>>>> 2) bitbake core-image-sato with MACHINE=crownbay
> >>>>>>
> >>>>>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
> >>>>>> image even if there is one in deploy/ipk/core2 and we have set the
> >>>>>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
> >>>>>> higher. This doesn't work for us, for example, what the crownbay need is
> >>>>>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
> >>>>>
> >>>>> And this is working exactly as intended. Don't break opkg because your
> >>>>> hardware driver situation sucks.
> >>>>>
> >>>>> So: NAK on this patch.
> >>>>
> >>>> I think we do have a problem here. For example, the system is ignoring a
> >>>> PREFERRED_VERSION directive here
> >>>
> >>> A PREFERRED_VERSION set in a machine.conf, which is the wrong place.
> >>
> >> Its strongly not recommended. You can do it and if things are setup
> >> correctly, we do support machine specific alterations however...
> >>
> >>> Let's change the above build sequence to this:
> >>>
> >>> 1) MACHINE=qemux86 bitbake xserver-xorg
> >>> 2) MACHINE=othercore2machine bitbake xserver-xorg
> >>> 3) MACHINE=crownbay bitbake xserver-xorg
> >>>
> >>> Oh look, I get 1.11.2 on crownbay regardless of this patch.
> >>
> >> I've been assuming this xserver is marked as machine specific. If its
> >> not, that is a bug and we can fix that.
> >
> > the X server not machine specific and that is NOT a bug. The recipe for the DDX only works with a specific X version. The real problem here is that xf86-video-something (again, not machine specific, think pci cards) has a strict requirement on the x server version. This is now purely a DISTRO problem. A few possible solutions are:
> >
> > 1) Lock X to 1.9.3 for all compatible x86 archs in your DISTRO
> > 2) Lock X to 1.11.2 for all compatible x86 archs in your DISTRO, live with the crownbay breakage
> > 3) Only use a package manager with no notion of compatible archs (dpkg) or with strong version pinning (rpm)
> > 4) Be really careful with the build sequence and structure your feed configs to not have compatible archs in their feed URIs
> > 5) remove 'x11' from DISTRO_FEATURES
> >
> > This is a point where you cannot make everybody happy. But as you say below, let's decouple the above problem from the arch vs version discussion.
> >
> >>>> by building one thing and then
> >>>> installing another. We're also inconsistent between the dpkg/rpm and
> >>>> opkg backends. There is therefore definitely some kind of user
> >>>> experience issue at stake here since this behaviour is not obvious,
> >>>> expected or particularly correct.
> >>>>
> >>>> The fact the example is hardware related is not particularly relevant,
> >>>> its the bigger picture I worry about
> >>>
> >>> I also worry about the bigger picture, especially about what Martin
> >>> said about this breaking feeds.
> >>
> >> As far as I understood Martin's comments, this actually helps avoid some
> >> of the issues he's been experiencing with feeds?
> >
> > No, it allows you to add a package-of-death. Example:
> >
> > broken-app_1.0_machine.ipk is in the feeds, being machine specific
> > broken-apps author fixes the machine specific bits properly
> > broken-app_2.0_i586.ipk hits the feeds and is not picked up.
> 
> I think that the broken-app_2.0_i586.ipk came unexpectedly,
> there was a broken-app_1.0_machine.ipk (maybe also broken-app_1.0_i586.ipk,
> but it didn't matter), if it has been fixed to version 2.0, then there should
> be also a broken-app_2.0_machine.ipk (and it would be picked up).

If broken-app developers are now detecting machine capabilities (or
whatever) since 2.0 in runtime, then we don't need to build broken-app
for every single machine we support, so changing PACKAGE_ARCH from
MACHINE_ARCH to TUNE_PKGARCH is right thing to do to save compile time
and feed disk space.

Same use-case broken by this patch reported here:
http://lists.linuxtogo.org/pipermail/openembedded-core/2012-May/023154.html

Cheers,

> 
> // Robert
> 
> >
> >> Martin has a problem where machines are ending up with unoptimised
> >> versions of code on them and it would be good to fix opkg behaviour to
> >> do what people expect...
> >
> > Yes, feed updates are not atomic, but this patch breaks moving packages between archs like the broken-app example above. In angstrom a machine doesn't see feeds with compatible archs at runtime, e.g. beagleboard sees beagleboard, armv7a and noarch URIs. This reduces these kind of problems to images built with OE. I must note that this setup for feed URIs was done out of bandwidth, storage and ram concerns (good old ipkg), not to avoid Martins problems :)
> >
> > regards,
> >
> > Koen
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
> >
> >
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: [PATCH 1/1] opkg 0.1.8: respect to the arch when choose the alternatives

[Koen Kooi]

Op 4 jun. 2012, om 12:39 heeft Martin Jansa het volgende geschreven:

> On Mon, Jun 04, 2012 at 05:31:26PM +0800, Robert Yang wrote:
>> 
>> 
>> On 06/01/2012 06:35 PM, Koen Kooi wrote:
>>> 
>>> Op 1 jun. 2012, om 12:02 heeft Richard Purdie het volgende geschreven:
>>> 
>>>> On Fri, 2012-06-01 at 11:04 +0200, Koen Kooi wrote:
>>>>> Op 1 jun. 2012, om 10:17 heeft Richard Purdie het volgende geschreven:
>>>>> 
>>>>>> On Thu, 2012-05-31 at 17:01 +0200, Koen Kooi wrote:
>>>>>>> Op 31 mei 2012, om 16:13 heeft Robert Yang het volgende geschreven:
>>>>>>> 
>>>>>>>> There is a bug if we:
>>>>>>>> 1) bitbake core-image-sato-sdk MACHINE=qemux86
>>>>>>>> 2) bitbake core-image-sato with MACHINE=crownbay
>>>>>>>> 
>>>>>>>> Then several pkgs in deploy/ipk/i586 would be installed to crownbay's
>>>>>>>> image even if there is one in deploy/ipk/core2 and we have set the
>>>>>>>> core2's priority higher than i586, when the version in deploy/ipk/i586 is
>>>>>>>> higher. This doesn't work for us, for example, what the crownbay need is
>>>>>>>> xserver-xorg-1.9.3, but it installs xserver-xorg-1.11.2.
>>>>>>> 
>>>>>>> And this is working exactly as intended. Don't break opkg because your
>>>>>>> hardware driver situation sucks.
>>>>>>> 
>>>>>>> So: NAK on this patch.
>>>>>> 
>>>>>> I think we do have a problem here. For example, the system is ignoring a
>>>>>> PREFERRED_VERSION directive here
>>>>> 
>>>>> A PREFERRED_VERSION set in a machine.conf, which is the wrong place.
>>>> 
>>>> Its strongly not recommended. You can do it and if things are setup
>>>> correctly, we do support machine specific alterations however...
>>>> 
>>>>> Let's change the above build sequence to this:
>>>>> 
>>>>> 1) MACHINE=qemux86 bitbake xserver-xorg
>>>>> 2) MACHINE=othercore2machine bitbake xserver-xorg
>>>>> 3) MACHINE=crownbay bitbake xserver-xorg
>>>>> 
>>>>> Oh look, I get 1.11.2 on crownbay regardless of this patch.
>>>> 
>>>> I've been assuming this xserver is marked as machine specific. If its
>>>> not, that is a bug and we can fix that.
>>> 
>>> the X server not machine specific and that is NOT a bug. The recipe for the DDX only works with a specific X version. The real problem here is that xf86-video-something (again, not machine specific, think pci cards) has a strict requirement on the x server version. This is now purely a DISTRO problem. A few possible solutions are:
>>> 
>>> 1) Lock X to 1.9.3 for all compatible x86 archs in your DISTRO
>>> 2) Lock X to 1.11.2 for all compatible x86 archs in your DISTRO, live with the crownbay breakage
>>> 3) Only use a package manager with no notion of compatible archs (dpkg) or with strong version pinning (rpm)
>>> 4) Be really careful with the build sequence and structure your feed configs to not have compatible archs in their feed URIs
>>> 5) remove 'x11' from DISTRO_FEATURES
>>> 
>>> This is a point where you cannot make everybody happy. But as you say below, let's decouple the above problem from the arch vs version discussion.
>>> 
>>>>>> by building one thing and then
>>>>>> installing another. We're also inconsistent between the dpkg/rpm and
>>>>>> opkg backends. There is therefore definitely some kind of user
>>>>>> experience issue at stake here since this behaviour is not obvious,
>>>>>> expected or particularly correct.
>>>>>> 
>>>>>> The fact the example is hardware related is not particularly relevant,
>>>>>> its the bigger picture I worry about
>>>>> 
>>>>> I also worry about the bigger picture, especially about what Martin
>>>>> said about this breaking feeds.
>>>> 
>>>> As far as I understood Martin's comments, this actually helps avoid some
>>>> of the issues he's been experiencing with feeds?
>>> 
>>> No, it allows you to add a package-of-death. Example:
>>> 
>>> broken-app_1.0_machine.ipk is in the feeds, being machine specific
>>> broken-apps author fixes the machine specific bits properly
>>> broken-app_2.0_i586.ipk hits the feeds and is not picked up.
>> 
>> I think that the broken-app_2.0_i586.ipk came unexpectedly,
>> there was a broken-app_1.0_machine.ipk (maybe also broken-app_1.0_i586.ipk,
>> but it didn't matter), if it has been fixed to version 2.0, then there should
>> be also a broken-app_2.0_machine.ipk (and it would be picked up).
> 
> If broken-app developers are now detecting machine capabilities (or
> whatever) since 2.0 in runtime, then we don't need to build broken-app
> for every single machine we support, so changing PACKAGE_ARCH from
> MACHINE_ARCH to TUNE_PKGARCH is right thing to do to save compile time
> and feed disk space.

Exactly

> Same use-case broken by this patch reported here:
> http://lists.linuxtogo.org/pipermail/openembedded-core/2012-May/023154.html

I borrowed heavily from your post since both Richard and Robert don't seem to see the damage this will do to feeds when a package/recipe changes archs.

regards,

Koen