Conflicting providers for ssh/sshd (dropbear and openssh)

Conflicting providers for ssh/sshd (dropbear and openssh)

[Khem Raj]

ERROR: Multiple .bb files are due to be built which each provide ssh
(/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/dropbear/dropbear_0.52.bb
/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-connectivity/openssh/openssh_5.8p2.bb).
 This usually means one provides something the other doesn't and should.
ERROR: Multiple .bb files are due to be built which each provide sshd
(/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/dropbear/dropbear_0.52.bb
/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-connectivity/openssh/openssh_5.8p2.bb).
 This usually means one provides something the other doesn't and should.

reason is that dropbear only provides ssh and sshd packages openssh
provides a few more e.g. openssh-sftp-server which is demanded by
some images and at same time wants dropbear to provide sshd and ssh

Now both recipes are to be built but are in contention for providing
ssh and sshd

How to solve this problem. ? Are other packages that openssh provides
strictly depending on ssh/sshd from openssh ? or will they work on
any ssh/sshd ?

If they are independent then may be the openssh recipe should be
divided into openssh-ssh and openssh-rest so one can use openssh
provided daemon or dropbear provided as they wish

Thanks
-Khem

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Scott Garman]

On 06/28/2011 03:41 PM, Khem Raj wrote:
> ERROR: Multiple .bb files are due to be built which each provide ssh
> (/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/dropbear/dropbear_0.52.bb
> /home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-connectivity/openssh/openssh_5.8p2.bb).
>   This usually means one provides something the other doesn't and should.
> ERROR: Multiple .bb files are due to be built which each provide sshd
> (/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/dropbear/dropbear_0.52.bb
> /home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-connectivity/openssh/openssh_5.8p2.bb).
>   This usually means one provides something the other doesn't and should.
>
> reason is that dropbear only provides ssh and sshd packages openssh
> provides a few more e.g. openssh-sftp-server which is demanded by
> some images and at same time wants dropbear to provide sshd and ssh
>
> Now both recipes are to be built but are in contention for providing
> ssh and sshd
>
> How to solve this problem. ? Are other packages that openssh provides
> strictly depending on ssh/sshd from openssh ? or will they work on
> any ssh/sshd ?
>
> If they are independent then may be the openssh recipe should be
> divided into openssh-ssh and openssh-rest so one can use openssh
> provided daemon or dropbear provided as they wish

I've run into that and have been wondering how to resolve it as well.

If there are currently existing images which are pulling in dropbear's 
ssh and sshd while using openssh's sftp-server, then that would imply 
they can work independently, even though that combo seems like an 
aggressive space optimization that I would tend to avoid.

Scott

-- 
Scott Garman
Embedded Linux Engineer - Yocto Project
Intel Open Source Technology Center

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Khem Raj]

On Jun 28, 2011, at 3:51 PM, Scott Garman <scott.a.garman@intel.com> wrote:

> On 06/28/2011 03:41 PM, Khem Raj wrote:
>> ERROR: Multiple .bb files are due to be built which each provide ssh
>> (/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/dropbear/dropbear_0.52.bb
>> /home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-connectivity/openssh/openssh_5.8p2.bb).
>>  This usually means one provides something the other doesn't and should.
>> ERROR: Multiple .bb files are due to be built which each provide sshd
>> (/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/dropbear/dropbear_0.52.bb
>> /home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-connectivity/openssh/openssh_5.8p2.bb).
>>  This usually means one provides something the other doesn't and should.
>> 
>> reason is that dropbear only provides ssh and sshd packages openssh
>> provides a few more e.g. openssh-sftp-server which is demanded by
>> some images and at same time wants dropbear to provide sshd and ssh
>> 
>> Now both recipes are to be built but are in contention for providing
>> ssh and sshd
>> 
>> How to solve this problem. ? Are other packages that openssh provides
>> strictly depending on ssh/sshd from openssh ? or will they work on
>> any ssh/sshd ?
>> 
>> If they are independent then may be the openssh recipe should be
>> divided into openssh-ssh and openssh-rest so one can use openssh
>> provided daemon or dropbear provided as they wish
> 
> I've run into that and have been wondering how to resolve it as well.
> 
> If there are currently existing images which are pulling in dropbear's ssh and sshd while using openssh's sftp-server, then that would imply they can work independently, even though that combo seems like an aggressive space optimization that I would tend to avoid.

yes if you build console-image with angstrom you will see it wanting sftp-server and dr bear for rest of ssh needs
> 
> Scott
> 
> -- 
> Scott Garman
> Embedded Linux Engineer - Yocto Project
> Intel Open Source Technology Center
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Graeme Gregory]

On 06/29/2011 12:50 AM, Khem Raj wrote:
>
> On Jun 28, 2011, at 3:51 PM, Scott Garman <scott.a.garman@intel.com> wrote:
>
>> On 06/28/2011 03:41 PM, Khem Raj wrote:
>>> ERROR: Multiple .bb files are due to be built which each provide ssh
>>> (/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/dropbear/dropbear_0.52.bb
>>> /home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-connectivity/openssh/openssh_5.8p2.bb).
>>>  This usually means one provides something the other doesn't and should.
>>> ERROR: Multiple .bb files are due to be built which each provide sshd
>>> (/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/dropbear/dropbear_0.52.bb
>>> /home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-connectivity/openssh/openssh_5.8p2.bb).
>>>  This usually means one provides something the other doesn't and should.
>>>
>>> reason is that dropbear only provides ssh and sshd packages openssh
>>> provides a few more e.g. openssh-sftp-server which is demanded by
>>> some images and at same time wants dropbear to provide sshd and ssh
>>>
>>> Now both recipes are to be built but are in contention for providing
>>> ssh and sshd
>>>
>>> How to solve this problem. ? Are other packages that openssh provides
>>> strictly depending on ssh/sshd from openssh ? or will they work on
>>> any ssh/sshd ?
>>>
>>> If they are independent then may be the openssh recipe should be
>>> divided into openssh-ssh and openssh-rest so one can use openssh
>>> provided daemon or dropbear provided as they wish
>> I've run into that and have been wondering how to resolve it as well.
>>
>> If there are currently existing images which are pulling in dropbear's ssh and sshd while using openssh's sftp-server, then that would imply they can work independently, even though that combo seems like an aggressive space optimization that I would tend to avoid.
> yes if you build console-image with angstrom you will see it wanting sftp-server and dr bear for rest of ssh needs
>
sftp-server is just a really small ftp like program that gets run as the
shell when sftp connects instead of ssh. It has no dependency on any
openssh code or even openssl. This is the reason why the dropbear
developer decided to just use it instead of writing his own exact work
alike.

I was the one that added it to Angstrom images as I really like to sftp
out of the box.

Graeme

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Scott Garman]

On 06/28/2011 03:41 PM, Khem Raj wrote:
> If they are independent then may be the openssh recipe should be
> divided into openssh-ssh and openssh-rest so one can use openssh
> provided daemon or dropbear provided as they wish

I'm happy to do this, but won't be able to get to it until early next 
week. In the meantime Yocto bug #1009 is tracking this issue.

Scott

-- 
Scott Garman
Embedded Linux Engineer - Yocto Project
Intel Open Source Technology Center

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Khem Raj]

On 06/28/2011 04:53 PM, Graeme Gregory wrote:
> On 06/29/2011 12:50 AM, Khem Raj wrote:
>>
>> On Jun 28, 2011, at 3:51 PM, Scott Garman<scott.a.garman@intel.com>  wrote:
>>
>>> On 06/28/2011 03:41 PM, Khem Raj wrote:
>>>> ERROR: Multiple .bb files are due to be built which each provide ssh
>>>> (/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/dropbear/dropbear_0.52.bb
>>>> /home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-connectivity/openssh/openssh_5.8p2.bb).
>>>>   This usually means one provides something the other doesn't and should.
>>>> ERROR: Multiple .bb files are due to be built which each provide sshd
>>>> (/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/dropbear/dropbear_0.52.bb
>>>> /home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-connectivity/openssh/openssh_5.8p2.bb).
>>>>   This usually means one provides something the other doesn't and should.
>>>>
>>>> reason is that dropbear only provides ssh and sshd packages openssh
>>>> provides a few more e.g. openssh-sftp-server which is demanded by
>>>> some images and at same time wants dropbear to provide sshd and ssh
>>>>
>>>> Now both recipes are to be built but are in contention for providing
>>>> ssh and sshd
>>>>
>>>> How to solve this problem. ? Are other packages that openssh provides
>>>> strictly depending on ssh/sshd from openssh ? or will they work on
>>>> any ssh/sshd ?
>>>>
>>>> If they are independent then may be the openssh recipe should be
>>>> divided into openssh-ssh and openssh-rest so one can use openssh
>>>> provided daemon or dropbear provided as they wish
>>> I've run into that and have been wondering how to resolve it as well.
>>>
>>> If there are currently existing images which are pulling in dropbear's ssh and sshd while using openssh's sftp-server, then that would imply they can work independently, even though that combo seems like an aggressive space optimization that I would tend to avoid.
>> yes if you build console-image with angstrom you will see it wanting sftp-server and dr bear for rest of ssh needs
>>
> sftp-server is just a really small ftp like program that gets run as the
> shell when sftp connects instead of ssh. It has no dependency on any
> openssh code or even openssl.

ok that solves my question about deps on ssh/ssh from openssh. So it 
seems we can refactor the openssh recipe


  This is the reason why the dropbear
> developer decided to just use it instead of writing his own exact work
> alike.
>
> I was the one that added it to Angstrom images as I really like to sftp
> out of the box.
>
> Graeme
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Anders Darander]

On Wed, Jun 29, 2011 at 00:51, Scott Garman <scott.a.garman@intel.com> wrote:
> On 06/28/2011 03:41 PM, Khem Raj wrote:
>> reason is that dropbear only provides ssh and sshd packages openssh
>> provides a few more e.g. openssh-sftp-server which is demanded by
>> some images and at same time wants dropbear to provide sshd and ssh
>>
>> Now both recipes are to be built but are in contention for providing
>> ssh and sshd
>>
>> How to solve this problem. ? Are other packages that openssh provides
>> strictly depending on ssh/sshd from openssh ? or will they work on
>> any ssh/sshd ?
>>
>> If they are independent then may be the openssh recipe should be
>> divided into openssh-ssh and openssh-rest so one can use openssh
>> provided daemon or dropbear provided as they wish

Sounds good. I'd probably suggest openssh-ssh and openssh-sftp, where the
latter one only includes what's necessary for the sftp-server to work. (Or maybe
that was similar to what you meant?)

> I've run into that and have been wondering how to resolve it as well.
>
> If there are currently existing images which are pulling in dropbear's ssh
> and sshd while using openssh's sftp-server, then that would imply they can
> work independently, even though that combo seems like an aggressive space
> optimization that I would tend to avoid.

I wouldn't say that it is a very common practice, but it certainly
isn't too uncommon.
I've seen it a number of times, and also used it in a few occasions.

Regards,
Anders

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Koen Kooi]

Op 29 jun 2011, om 00:41 heeft Khem Raj het volgende geschreven:
> If they are independent then may be the openssh recipe should be
> divided into openssh-ssh and openssh-rest so one can use openssh
> provided daemon or dropbear provided as they wish

Dividing the openssl recipe would gain us little and the gains would be only for the power companies since you'd have to build openssh twice to get both sftp and ssh. The decrease in build time for only sftp is neglible.

The real problem is the everything-is-a-distro-choice history we inherited from Poky.

regards,

Koen

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Anders Darander]

On Wed, Jun 29, 2011 at 10:24, Koen Kooi <koen@dominion.thruhere.net> wrote:
> Op 29 jun 2011, om 00:41 heeft Khem Raj het volgende geschreven:
>> If they are independent then may be the openssh recipe should be
>> divided into openssh-ssh and openssh-rest so one can use openssh
>> provided daemon or dropbear provided as they wish
>
> Dividing the openssl recipe would gain us little and the gains would be only for the power companies since you'd have to build openssh twice to get both sftp and ssh. The decrease in build time for only sftp is neglible.

Hm, speaking against what I've often been advocating (reducing build
time by factoring out dependenies etc)...

I think the simplest and most straightforward solution is to just
split the packaging into
openssh-ssh and openssh-sftp, where openssh-sftp packages just what is
needed for handling
the sftp-server in cooperation with dropbear. It could possibly also
include the sftp-client if
desired/needed.

The openssh-ssh package could then depend on the openssh-sftp package;
then there would
be no difference today for the distros using the complete openssh package.

I read the
>> If they are independent then may be the openssh recipe should be
>> divided into openssh-ssh and openssh-rest
part as just splitting the package. But now when I re-read it, it could very
well have implied creating two recipes; which I agree wouldn't be the best
option.

Regards,
Anders

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Koen Kooi]

Op 29 jun 2011, om 10:50 heeft Anders Darander het volgende geschreven:

> On Wed, Jun 29, 2011 at 10:24, Koen Kooi <koen@dominion.thruhere.net> wrote:
>> Op 29 jun 2011, om 00:41 heeft Khem Raj het volgende geschreven:
>>> If they are independent then may be the openssh recipe should be
>>> divided into openssh-ssh and openssh-rest so one can use openssh
>>> provided daemon or dropbear provided as they wish
>> 
>> Dividing the openssl recipe would gain us little and the gains would be only for the power companies since you'd have to build openssh twice to get both sftp and ssh. The decrease in build time for only sftp is neglible.
> 
> Hm, speaking against what I've often been advocating (reducing build
> time by factoring out dependenies etc)...
> 
> I think the simplest and most straightforward solution is to just
> split the packaging into
> openssh-ssh and openssh-sftp, where openssh-sftp packages just what is
> needed for handling
> the sftp-server in cooperation with dropbear. It could possibly also
> include the sftp-client if
> desired/needed.

That's already the case now. The problem is the PROVIDES overlap since the Poky people decided a distro could only have one true ssh implementation instead of choosing it per image. So if your distro doesn't set the PREFERRED_PROVIDER_sshd you get those nagging messages during parsing that scare users and make consultants rich.

OE .dev isn't a lot better with the misguided DISTRO_SSH_DAEMON, but at least it doesn't show those nag messages.

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Graeme Gregory]

On 06/29/2011 09:50 AM, Anders Darander wrote:
> On Wed, Jun 29, 2011 at 10:24, Koen Kooi <koen@dominion.thruhere.net> wrote:
>> Op 29 jun 2011, om 00:41 heeft Khem Raj het volgende geschreven:
>>> If they are independent then may be the openssh recipe should be
>>> divided into openssh-ssh and openssh-rest so one can use openssh
>>> provided daemon or dropbear provided as they wish
>> Dividing the openssl recipe would gain us little and the gains would be only for the power companies since you'd have to build openssh twice to get both sftp and ssh. The decrease in build time for only sftp is neglible.
> Hm, speaking against what I've often been advocating (reducing build
> time by factoring out dependenies etc)...
>
> I think the simplest and most straightforward solution is to just
> split the packaging into
> openssh-ssh and openssh-sftp, where openssh-sftp packages just what is
> needed for handling
> the sftp-server in cooperation with dropbear. It could possibly also
> include the sftp-client if
> desired/needed.
>
> The openssh-ssh package could then depend on the openssh-sftp package;
> then there would
> be no difference today for the distros using the complete openssh package.
>
> I read the
>>> If they are independent then may be the openssh recipe should be
>>> divided into openssh-ssh and openssh-rest
> part as just splitting the package. But now when I re-read it, it could very
> well have implied creating two recipes; which I agree wouldn't be the best
> option.
>
The package in OE has been split for a long long time since I first
discovered about dropbear being about to use sftp-server.

Graeme

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Phil Blundell]

On Wed, 2011-06-29 at 10:56 +0200, Koen Kooi wrote:
> That's already the case now. The problem is the PROVIDES overlap since
> the Poky people decided a distro could only have one true ssh
> implementation instead of choosing it per image. So if your distro
> doesn't set the PREFERRED_PROVIDER_sshd you get those nagging messages
> during parsing that scare users and make consultants rich.
> 
> OE .dev isn't a lot better with the misguided DISTRO_SSH_DAEMON, but
> at least it doesn't show those nag messages.

Fundamentally I think it is just a bug in bitbake that it makes such a
fuss about overlapping PROVIDES.  It's not unreasonable for both openssh
and dropbear to be PROVIDEing something like virtual/ssh-daemon (and
indeed RPROVIDEing an equivalent) but, as you say, any given distro is
perfectly entitled to want to build both of them and ship them in
different images and/or feeds.

I guess what bitbake is really trying to warn about is recipes which
will install conflicting files into the sysroot.  Obviously in a future
utopia of per-recipe sysroot construction this would be a non-issue, but
even with today's level of technology I think it would be fairly easy
for us to detect when a collision actually happens and issue a sensible
diagnostic at that point.  That would allow the offending ERROR to be
removed without causing any real regression.

p.

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Anders Darander]

On Wed, Jun 29, 2011 at 10:56, Koen Kooi <koen@dominion.thruhere.net> wrote:
> Op 29 jun 2011, om 10:50 heeft Anders Darander het volgende geschreven:
>> On Wed, Jun 29, 2011 at 10:24, Koen Kooi <koen@dominion.thruhere.net> wrote:
>>> Op 29 jun 2011, om 00:41 heeft Khem Raj het volgende geschreven:
>>>> If they are independent then may be the openssh recipe should be
>>>> divided into openssh-ssh and openssh-rest so one can use openssh
>>>> provided daemon or dropbear provided as they wish
>>>
>>> Dividing the openssl recipe would gain us little and the gains would be only for the power companies since you'd have to build openssh twice to get both sftp and ssh. The decrease in build time for only sftp is neglible.
>>
>> Hm, speaking against what I've often been advocating (reducing build
>> time by factoring out dependenies etc)...
>>
>> I think the simplest and most straightforward solution is to just
>> split the packaging into
>> openssh-ssh and openssh-sftp, where openssh-sftp packages just what is
>> needed for handling
>> the sftp-server in cooperation with dropbear. It could possibly also
>> include the sftp-client if
>> desired/needed.
>
> That's already the case now. The problem is the PROVIDES overlap since the Poky people decided a distro could only have one true ssh implementation instead of choosing it per image. So if your distro doesn't set the PREFERRED_PROVIDER_sshd you get those nagging messages during parsing that scare users and make consultants rich.
>
> OE .dev isn't a lot better with the misguided DISTRO_SSH_DAEMON, but at least it doesn't show those nag messages.

Ah, I see. I didn't check it out, just assumed something...

Then I fully understand what the problem case is.

Anders

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Richard Purdie]

On Wed, 2011-06-29 at 10:08 +0100, Phil Blundell wrote:
> On Wed, 2011-06-29 at 10:56 +0200, Koen Kooi wrote:
> > That's already the case now. The problem is the PROVIDES overlap since
> > the Poky people decided a distro could only have one true ssh
> > implementation instead of choosing it per image. So if your distro
> > doesn't set the PREFERRED_PROVIDER_sshd you get those nagging messages
> > during parsing that scare users and make consultants rich.
> > 
> > OE .dev isn't a lot better with the misguided DISTRO_SSH_DAEMON, but
> > at least it doesn't show those nag messages.
> 
> Fundamentally I think it is just a bug in bitbake that it makes such a
> fuss about overlapping PROVIDES.  It's not unreasonable for both openssh
> and dropbear to be PROVIDEing something like virtual/ssh-daemon (and
> indeed RPROVIDEing an equivalent) but, as you say, any given distro is
> perfectly entitled to want to build both of them and ship them in
> different images and/or feeds.
> 
> I guess what bitbake is really trying to warn about is recipes which
> will install conflicting files into the sysroot.  Obviously in a future
> utopia of per-recipe sysroot construction this would be a non-issue, but
> even with today's level of technology I think it would be fairly easy
> for us to detect when a collision actually happens and issue a sensible
> diagnostic at that point.  That would allow the offending ERROR to be
> removed without causing any real regression.

Speaking as the person who likely added this code in the first place,
its not quite this simple. There are two problem cases:

a) When we have several recipes like external-toolchain, glibc, eglibc
and uclibc all of which provide virtual/libc. If something else in the
build wants say eglibc-locale-foo but the preferred provider of
virtual/libc is uclibc then what bitbake is trying to warn about is that
it the user isn't going to get what they expect as both uclibc and
eglibc could end up being built.

We used to end up in a mess where bitbake could build the
external-toolchain recipe and some other libcs in parallel resulting an
a what could only be described as a mess. These days this doesn't happen
so much due to bitbake spitting out its dummy earlier on. It is actually
quite hard to detect this problem generically.

b) There is also the case where two recipes might generate the same
package. There is also some code in bitbake which at least tries to flag
problems like that from the point of view of multiple runtime providers.


So the code does actually stop some pretty nasty breakage from happening
but it isn't perfect and if we can find better ways, by all means...

Cheers,

Richard

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Phil Blundell]

On Wed, 2011-06-29 at 10:42 +0100, Richard Purdie wrote:
> Speaking as the person who likely added this code in the first place,
> its not quite this simple. There are two problem cases:
> 
> a) When we have several recipes like external-toolchain, glibc, eglibc
> and uclibc all of which provide virtual/libc. If something else in the
> build wants say eglibc-locale-foo but the preferred provider of
> virtual/libc is uclibc then what bitbake is trying to warn about is that
> it the user isn't going to get what they expect as both uclibc and
> eglibc could end up being built.

In the particular case of uclibc vs eglibc, this isn't going to happen
because the COMPATIBLE_HOST logic will only ever admit one or the other
to the set of allowed recipes.  And, in the general case of arbitrary
packages, the point I was trying to make earlier is that the user might
actually _want_ to build two things which happen to both provide the
same virtual, and shouldn't be prevented (or even necessarily
discouraged) from doing so.

If there are other cases like the eglibc/uclibc thing which aren't
"intrinsically safe" in the same way then I guess the right way to solve
that is via some kind of (R)CONFLICTS declaration and smarter dependency
processing within bitbake.

> b) There is also the case where two recipes might generate the same
> package. There is also some code in bitbake which at least tries to flag
> problems like that from the point of view of multiple runtime providers.

As with the sysroot thing, I think the place to detect and diagnose that
is at the point where the packages are generated.  Or, alternatively
(but slightly less robustly) you could just scan PACKAGES for all the
recipes in the runqueue and look for collisions there, which would allow
you to detect at least some problems sooner.  I don't think the list of
PROVIDES is sufficiently reliable to use for this purpose since you can
easily have both false positives and false negatives.

p.

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Richard Purdie]

On Wed, 2011-06-29 at 10:51 +0100, Phil Blundell wrote:
> On Wed, 2011-06-29 at 10:42 +0100, Richard Purdie wrote:
> > Speaking as the person who likely added this code in the first place,
> > its not quite this simple. There are two problem cases:
> > 
> > a) When we have several recipes like external-toolchain, glibc, eglibc
> > and uclibc all of which provide virtual/libc. If something else in the
> > build wants say eglibc-locale-foo but the preferred provider of
> > virtual/libc is uclibc then what bitbake is trying to warn about is that
> > it the user isn't going to get what they expect as both uclibc and
> > eglibc could end up being built.
> 
> In the particular case of uclibc vs eglibc, this isn't going to happen
> because the COMPATIBLE_HOST logic will only ever admit one or the other
> to the set of allowed recipes.  And, in the general case of arbitrary
> packages, the point I was trying to make earlier is that the user might
> actually _want_ to build two things which happen to both provide the
> same virtual, and shouldn't be prevented (or even necessarily
> discouraged) from doing so.
> 
> If there are other cases like the eglibc/uclibc thing which aren't
> "intrinsically safe" in the same way

glibc/eglibc?

external-toolchain?

This area used to be a world of pain for users as bitbake would merrily
go and build a ton of conflicting stuff and the user would be left with
no idea what was going on.

Before that code was in bitbake, it was a major frustration of users and
whilst there are issues like the ssh one, in general the current code
has done orders of magnitude more good than harm. I believe there is
also a whitelist variable to say "we really don't mind multiple
providers of X" too so there may actually be an already existing way to
work around this issue.

>  then I guess the right way to solve
> that is via some kind of (R)CONFLICTS declaration and smarter dependency
> processing within bitbake.

Likely yes but the trouble is you then have to explicitly mark two
things as conflicting so as soon as you add something new to the mix,
the mechanism doesn't work.

This also means adding *CONFLICTS support to bitbake which is something
we've wanted to do for a long time but never got around to.

> > b) There is also the case where two recipes might generate the same
> > package. There is also some code in bitbake which at least tries to flag
> > problems like that from the point of view of multiple runtime providers.
> 
> As with the sysroot thing, I think the place to detect and diagnose that
> is at the point where the packages are generated.  Or, alternatively
> (but slightly less robustly) you could just scan PACKAGES for all the
> recipes in the runqueue and look for collisions there, which would allow
> you to detect at least some problems sooner.  I don't think the list of
> PROVIDES is sufficiently reliable to use for this purpose since you can
> easily have both false positives and false negatives.

I don't disagree but as things stand PROVIDES has actually been vert
helpful to users in general, certainly its better than doing nothing.

Cheers,

Richard

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Scott Garman]

On 06/29/2011 01:56 AM, Koen Kooi wrote:
>
> Op 29 jun 2011, om 10:50 heeft Anders Darander het volgende
> geschreven:
>
>> On Wed, Jun 29, 2011 at 10:24, Koen
>> Kooi<koen@dominion.thruhere.net>  wrote:
>>> Op 29 jun 2011, om 00:41 heeft Khem Raj het volgende geschreven:
>>>> If they are independent then may be the openssh recipe should
>>>> be divided into openssh-ssh and openssh-rest so one can use
>>>> openssh provided daemon or dropbear provided as they wish
>>>
>>> Dividing the openssl recipe would gain us little and the gains
>>> would be only for the power companies since you'd have to build
>>> openssh twice to get both sftp and ssh. The decrease in build
>>> time for only sftp is neglible.
>>
>> Hm, speaking against what I've often been advocating (reducing
>> build time by factoring out dependenies etc)...
>>
>> I think the simplest and most straightforward solution is to just
>> split the packaging into openssh-ssh and openssh-sftp, where
>> openssh-sftp packages just what is needed for handling the
>> sftp-server in cooperation with dropbear. It could possibly also
>> include the sftp-client if desired/needed.
>
> That's already the case now. The problem is the PROVIDES overlap
> since the Poky people decided a distro could only have one true ssh
> implementation instead of choosing it per image. So if your distro
> doesn't set the PREFERRED_PROVIDER_sshd you get those nagging
> messages during parsing that scare users and make consultants rich.
>
> OE .dev isn't a lot better with the misguided DISTRO_SSH_DAEMON, but
> at least it doesn't show those nag messages.

It's worth noting that one of the things I got into master just after 
Yocto 1.0 shipped was a refactoring of how ssh servers were specified. 
It no longer is a distro choice - we have task-core-ssh-openssh and 
task-core-ssh-dropbear that you add to IMAGE_FEATURES for your desired 
image.

Which makes me wonder what the consequences would be to simply remove 
the PROVIDES from the dropbear and openssh recipes?

Scott

-- 
Scott Garman
Embedded Linux Engineer - Yocto Project
Intel Open Source Technology Center

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Philip Balister]

On 06/29/2011 09:56 AM, Koen Kooi wrote:
>
> That's already the case now. The problem is the PROVIDES overlap since the Poky people decided a distro could only have one true ssh implementation instead of choosing it per image. So if your distro doesn't set the PREFERRED_PROVIDER_sshd you get those nagging messages during parsing that scare users and make consultants rich.
>

What's wrong with consultants getting rich?

Philip

Re: Conflicting providers for ssh/sshd (dropbear and openssh)

[Graeme Gregory]

On 07/03/2011 01:26 PM, Philip Balister wrote:
> On 06/29/2011 09:56 AM, Koen Kooi wrote:
>>
>> That's already the case now. The problem is the PROVIDES overlap
>> since the Poky people decided a distro could only have one true ssh
>> implementation instead of choosing it per image. So if your distro
>> doesn't set the PREFERRED_PROVIDER_sshd you get those nagging
>> messages during parsing that scare users and make consultants rich.
>>
>
> What's wrong with consultants getting rich?
>
I like this scheme :-)

Graeme