Default network connectivity sanity tests

Default network connectivity sanity tests

[Richard Purdie]

Currently the poky distro config tests for three different kinds of
network connectivity:

# The CONNECTIVITY_CHECK_URI's are used to test whether we can succesfully
# fetch from the network (and warn you if not). To disable the test set
# the variable to be empty.
CONNECTIVITY_CHECK_URIS ?= "git://git.yoctoproject.org/yocto-firewall-test;protocol=git;rev=HEAD \
                          https://eula-downloads.yoctoproject.org/index.php \
                          http://bugzilla.yoctoproject.org/report.cgi"

So this tests http, https and git. The theory goes that the project
mirrors should be a) preferred and b) be up to date in most cases so
that you'd only ever usually need http:// to run a build successfully.

There are some "windows of opportunity" when building from master where
this might not be true, for example, a kernel update has just been
pushed and the mirrors have not updated to the latest code. In this
scenario, bitbake would pull the mirror tarball, then updated it using
the git protocol.

I'm getting continued complaints that git is horrible to proxy behind
firewalls and this check continues to cause pain for people when in fact
the builds would probably work. These complains go so far as to say
we're ruining the reputation of the project by having this as people
leave after an initial bad experience. I'm reaching the point where I
become tempted to remove it.

Opinions?

One other option would be to remove it on release branches. Those should
be more stable and therefore shouldn't need this kind of check...

Cheers,

Richard

Re: Default network connectivity sanity tests

[Trevor Woerner]

Hi Richard,

On Thu, Sep 13, 2012 at 1:50 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> I'm getting continued complaints that git is horrible to proxy behind
> firewalls and this check continues to cause pain for people when in fact
> the builds would probably work.

I agree completely with your above statement. Thankfully I was able to
get git to work through the proxy which is setup where I work (so I'm
not affected by this), but only after a lot of work. However, even
though git works for me and can get through, it is extremely slow to
start, making it less preferred than http.

> One other option would be to remove it on release branches. Those should
> be more stable and therefore shouldn't need this kind of check...

Yes. Strangely enough I've had the same thoughts too!

In order for someone to be using the latest-and-greatest HEAD of the
yocto project, they would have to be using git to get the
latest-and-greatest, which implies git works for this person and maybe
even implies they have some sort of familiarity with using git. Plus
one would be more inclined to believe the HEAD of the yocto git
repositories might track the HEAD of the git repositories of the
various recipes as well.

But when someone downloads a yocto release via http it only
demonstrates http works so it probably isn't a good idea to assume a
whole bunch of other protocols work too. For example, take any yocto
release and inside we'll find a whole bunch of recipes. Are there any
cases where a yocto release is made but a recipe doesn't explicitly
specify the revision of a given component (i.e. uses git HEAD or some
random commit in a git repository) which is required? Ideally release
'H' of yocto would contain recipes which rely on specific releases of
various components (release x of eglibc, release y of busybox, release
z of libz...) all of which, ideally, someone could download from their
nearest GNU or sourceforge mirror.

I think someone (was it Tim Bird?) once mentioned how useful it would
be to be able to download a tarball of all the required packages,
place them in DL_DIR, then start a build with no connection to the
internet whatsoever (perhaps using different machines for the download
versus the build). That would be handy. Or, in the same way a user can
do " bitbake -c fetchall <image>" maybe it would be neat to have a
command which would list all the required components and how the build
expects to obtain said component (i.e. DL_DIR (if the component has
already been downloaded and ready to go) or a URI indicating where and
how bitbake would obtain it).

Best regards,
    Trevor