* Re: cannot re-use shared state cache between build hosts
2017-09-01 15:04 ` Andrea Galbusera
@ 2017-09-01 15:16 ` Maciej Borzęcki
2017-09-03 21:35 ` Paul Eggleton
2017-09-04 7:33 ` Patrick Ohly
2 siblings, 0 replies; 11+ messages in thread
From: Maciej Borzęcki @ 2017-09-01 15:16 UTC (permalink / raw)
To: Andrea Galbusera; +Cc: yocto@yoctoproject.org
On Fri, Sep 1, 2017 at 5:04 PM, Andrea Galbusera <gizero@gmail.com> wrote:
> Hi Maciej,
>
> On Fri, Sep 1, 2017 at 4:08 PM, Maciej Borzęcki <maciej.borzecki@rndity.com>
> wrote:
>>
>> On Fri, Sep 1, 2017 at 3:54 PM, Andrea Galbusera <gizero@gmail.com> wrote:
>> > Hi!
>> >
>> > I was trying to share sstate between different hosts, but the consumer
>> > build
>> > system seems to be unable to use re-use any sstate object. My scenario
>> > is
>> > setup as follows:
>> >
>> > * The cache was populated by a pristine qemux86 core-image-minimal build
>> > of
>> > morty. This was done in a crops/poky container (running in docker on
>> > Mac)
>> > * The cache was then served via HTTP
>>
>> Make sure that you use a decent HTTP server. Simple `python3 -m
>> http.server` will quickly choke when the mirror is being checked. Also
>> running bitbake -DDD -v makes investigating this much easier.
>
>
> To be honest, the current server was indeed setup with python's
> SimpleHTTPServer... As you suggest, I checked the verbose debug log and
> noticed what's happening behind the apparently happy "Checking sstate mirror
> object availability" step. After a first "SState: Successful fetch test for"
> that I see correctly served with 200 on the server side, tests for any other
> sstate object suddenly and systematically fail with logs like this:
>
> DEBUG: SState: Attempting to fetch
> file://7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz
> DEBUG: Searching for
> 7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz
> in paths:
> /home/vagrant/koan/morty/build/sstate-cache
> DEBUG: Defaulting to
> /home/vagrant/koan/morty/build/sstate-cache/7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz
> for 7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23b
> db86ac7ab32_package_qa.tgz
> DEBUG: Testing URL
> file://7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz
> DEBUG: For url ['file', '',
> '7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz',
> '', '', OrderedDict()] comparing ['file', '', '.*', '', '', OrderedDict()]
> to ['http', '192.168.33.1:8000', '
> /sstate-cache/PATH', '', '', OrderedDict([('downloadfilename', 'PATH')])]
> DEBUG: For url
> file://7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz
> returning
> http://192.168.33.1:8000/sstate-cache/7d/sstate%3Alibxml2%3Ai586-poky-linux%3A2.9.4%3Ar0%3Ai586%3A3%3A7da8fc
> 3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz;downloadfilename=7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz
> DEBUG: checkstatus: trying again
> DEBUG: checkstatus() urlopen failed: <urlopen error [Errno 9] Bad file
> descriptor>
> DEBUG: SState: Unsuccessful fetch test for
> file://7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz
>
> Nothing is reported server-side for any of these failures... As you
> recommend, I'll try to setup something more "decent" for the HTTP server and
> see if it helps.
Yeah, I think this has to do with HTTP keepalive. Anyways, caddy
worked just fine (https://caddyserver.com).
>
>
>>
>> > * The second host is a VM running Ubuntu 16.04 where I set
>> > SSTATE_MIRRORS to
>> > point to the hosted sstate cache like this:
>> >
>> > SSTATE_MIRRORS ?= "\
>> > file://.*
>> > http://192.168.33.1:8000/sstate-cache/PATH;downloadfilename=PATH"
>> >
>> > * I checked with curl that the VM can successfully get sstate objects
>> > from
>> > the server.
>> > * Then I start a new build (same metadata revisions, default
>> > configuration
>> > for core-image-minimal) and each and every task run from scratch with no
>> > sstate cache re-use.
>> >
>> > Here are the two configurations from bitbake and /etc/lsb-release files:
>> >
>> > On the container used to seed sstate cache:
>> >
>> > Build Configuration:
>> > BB_VERSION = "1.32.0"
>> > BUILD_SYS = "x86_64-linux"
>> > NATIVELSBSTRING = "universal"
>> > TARGET_SYS = "i586-poky-linux"
>> > MACHINE = "qemux86"
>> > DISTRO = "poky"
>> > DISTRO_VERSION = "2.2.2"
>> > TUNE_FEATURES = "m32 i586"
>> > TARGET_FPU = ""
>> > meta
>> > meta-poky
>> > meta-yocto-bsp = "morty:2a70e84643381eca0e7bf7928d4a3d56f9651128"
>> >
>> > $ cat /etc/lsb-release
>> > DISTRIB_ID=Ubuntu
>> > DISTRIB_RELEASE=16.04
>> > DISTRIB_CODENAME=xenial
>> > DISTRIB_DESCRIPTION="Ubuntu 16.04.2 LTS"
>> >
>> > On the VM that should consume the cache:
>> >
>> > Build Configuration:
>> > BB_VERSION = "1.32.0"
>> > BUILD_SYS = "x86_64-linux"
>> > NATIVELSBSTRING = "Ubuntu-16.04"
>> > TARGET_SYS = "i586-poky-linux"
>> > MACHINE = "qemux86"
>> > DISTRO = "poky"
>> > DISTRO_VERSION = "2.2.2"
>> > TUNE_FEATURES = "m32 i586"
>> > TARGET_FPU = ""
>> > meta
>> > meta-poky
>> > meta-yocto-bsp = "morty:2a70e84643381eca0e7bf7928d4a3d56f9651128"
>> >
>> > $ cat /etc/lsb-release
>> > DISTRIB_ID=Ubuntu
>> > DISTRIB_RELEASE=16.04
>> > DISTRIB_CODENAME=xenial
>> > DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"
>> >
>> >
>> > To me, the only differing bit that in my understanding can lead to
>> > sstate
>> > cache objects invalidation is the value of NATIVELSBSTRING which is
>> > "universal" inside the container and "Ubuntu-16.04". This sounds strange
>> > to
>> > me, since both underlying systems are Ubuntu 16.04 (although not exactly
>> > the
>> > same dot release) as confirmed by /etc/lsb-release contents.
>> >
>> > Is the different NATIVELSBSTRING the root cause for everything being
>> > re-built? If so, what's causing them being different in the end and what
>> > does "universal" exactly mean (to me it looks like a more generic and
>> > incluse term than any distro label, so I'm confused...)?
>> >
>> >
>> > --
>> > _______________________________________________
>> > yocto mailing list
>> > yocto@yoctoproject.org
>> > https://lists.yoctoproject.org/listinfo/yocto
>> >
>>
>>
>>
>> --
>> Maciej Borzecki
>> RnDity
>
>
--
Maciej Borzecki
RnDity
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: cannot re-use shared state cache between build hosts
2017-09-01 15:04 ` Andrea Galbusera
2017-09-01 15:16 ` Maciej Borzęcki
@ 2017-09-03 21:35 ` Paul Eggleton
2017-09-04 7:33 ` Patrick Ohly
2 siblings, 0 replies; 11+ messages in thread
From: Paul Eggleton @ 2017-09-03 21:35 UTC (permalink / raw)
To: Andrea Galbusera; +Cc: yocto
Hi Andrea,
On Saturday, 2 September 2017 3:04:00 AM NZST Andrea Galbusera wrote:
> To be honest, the current server was indeed setup with python's
> SimpleHTTPServer... As you suggest, I checked the verbose debug log and
> noticed what's happening behind the apparently happy "Checking sstate
> mirror object availability" step. After a first "SState: Successful fetch
> test for" that I see correctly served with 200 on the server side, tests
> for any other sstate object suddenly and systematically fail with logs like
> this:
>
> DEBUG: SState: Attempting to fetch file://7d/sstate:libxml2:i586-
> poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz
> DEBUG: Searching for 7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3:
> 7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz in paths:
> /home/vagrant/koan/morty/build/sstate-cache
> DEBUG: Defaulting to /home/vagrant/koan/morty/build/sstate-cache/7d/sstate:
> libxml2:i586-poky-linux:
2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz
> for 7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23b
> db86ac7ab32_package_qa.tgz
> DEBUG: Testing URL file://7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3:
> 7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz
> DEBUG: For url ['file', '', '7d/sstate:libxml2:i586-poky-
> linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz', '',
> '', OrderedDict()] comparing ['file', '', '.*', '', '', OrderedDict()] to
> ['http', '192.168.33.1:8000', '
> /sstate-cache/PATH', '', '', OrderedDict([('downloadfilename', 'PATH')])]
> DEBUG: For url file://7d/sstate:libxml2:i586-poky-linux:2.9.4:r0:i586:3:
> 7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz returning
> http://192.168.33.1:8000/sstate-cache/7d/sstate%3Alibxml2%3Ai586-poky-linux%
> 3A2.9.4%3Ar0%3Ai586%3A3%3A7da8fc
> 3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz;downloadfilename=7d/sstate:
> libxml2:i586-poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab
> 32_package_qa.tgz
> DEBUG: checkstatus: trying again
> DEBUG: checkstatus() urlopen failed: <urlopen error [Errno 9] Bad file
> descriptor>
> DEBUG: SState: Unsuccessful fetch test for file://7d/sstate:libxml2:i586-
> poky-linux:2.9.4:r0:i586:3:7da8fc3f7f5ed0102d23bdb86ac7ab32_package_qa.tgz
>
> Nothing is reported server-side for any of these failures... As you
> recommend, I'll try to setup something more "decent" for the HTTP server
> and see if it helps.
Right, FYI we were using plain SimpleHTTPServer in oe-selftest, and we had to
change it because it simply isn't reliable when you throw multiple concurrent
requests at it as sstate does. There's a ThreadingMixIn you can inherit (see
meta/lib/oeqa/utils/httpserver.py) but you might as well set up a proper
server if you're doing more than just testing.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: cannot re-use shared state cache between build hosts
2017-09-01 15:04 ` Andrea Galbusera
2017-09-01 15:16 ` Maciej Borzęcki
2017-09-03 21:35 ` Paul Eggleton
@ 2017-09-04 7:33 ` Patrick Ohly
2017-09-04 7:59 ` Andrea Galbusera
2 siblings, 1 reply; 11+ messages in thread
From: Patrick Ohly @ 2017-09-04 7:33 UTC (permalink / raw)
To: Andrea Galbusera, Maciej Borzęcki; +Cc: yocto@yoctoproject.org
On Fri, 2017-09-01 at 17:04 +0200, Andrea Galbusera wrote:
> Hi Maciej,
>
> On Fri, Sep 1, 2017 at 4:08 PM, Maciej Borzęcki <maciej.borzecki@rndi
> ty.com> wrote:
> > On Fri, Sep 1, 2017 at 3:54 PM, Andrea Galbusera <gizero@gmail.com>
> > wrote:
> > > Hi!
> > >
> > > I was trying to share sstate between different hosts, but the
> > consumer build
> > > system seems to be unable to use re-use any sstate object. My
> > scenario is
> > > setup as follows:
> > >
> > > * The cache was populated by a pristine qemux86 core-image-
> > minimal build of
> > > morty. This was done in a crops/poky container (running in docker
> > on Mac)
> > > * The cache was then served via HTTP
> >
> > Make sure that you use a decent HTTP server. Simple `python3 -m
> > http.server` will quickly choke when the mirror is being checked.
> > Also
> > running bitbake -DDD -v makes investigating this much easier.
>
> To be honest, the current server was indeed setup with python's
> SimpleHTTPServer... As you suggest, I checked the verbose debug log
> and noticed what's happening behind the apparently happy "Checking
> sstate mirror object availability" step. After a first "SState:
> Successful fetch test for" that I see correctly served with 200 on
> the server side, tests for any other sstate object suddenly and
> systematically fail with logs like this:
...
> DEBUG: checkstatus() urlopen failed: <urlopen error [Errno 9] Bad
> file descriptor>
More recent bitbake should not fail like that anymore. It's still
better to use an HTTP server that performs better, though.
commit 6fa07752bbd3ac345cd8617da49a70e0b2dd565f
Author: Patrick Ohly <patrick.ohly@intel.com>
Date: Mon Jul 17 15:25:10 2017 +0200
fetch2/wget.py: improve error handling during sstate check
When the sstate is accessed via HTTP, the existence check can fail due
to network issues, in which case bitbake silently continues without
sstate.
One such network issue is an HTTP server like Python's own SimpleHTTP
which closes the TCP connection despite an explicit "Keep-Alive" in
the HTTP request header. The server does that without a "close" in the
HTTP response header, so the socket remains in the connection cache,
leading to "urlopen failed: <urlopen error [Errno 9] Bad file
descriptor>" (only visible in "bitbake -D -D" output) when trying to
use the cached connection again.
The connection might also get closed for other reasons (proxy,
timeouts, etc.), so this is something that the client should be able
to handle.
This is achieved by checking for the error, removing the bad
connection, and letting the check_status() method try again with a new
connection. It is necessary to let the second attempt fail
permanently, because bad proxy setups have been observed to also lead
to such broken connections. In that case, we need to abort for real
after trying twice, otherwise a build would just hang forever.
[YOCTO #11782]
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: cannot re-use shared state cache between build hosts
2017-09-04 7:33 ` Patrick Ohly
@ 2017-09-04 7:59 ` Andrea Galbusera
2017-09-04 8:56 ` Paul Eggleton
0 siblings, 1 reply; 11+ messages in thread
From: Andrea Galbusera @ 2017-09-04 7:59 UTC (permalink / raw)
To: Patrick Ohly; +Cc: yocto@yoctoproject.org
[-- Attachment #1: Type: text/plain, Size: 3957 bytes --]
On Mon, Sep 4, 2017 at 9:33 AM, Patrick Ohly <patrick.ohly@intel.com> wrote:
> On Fri, 2017-09-01 at 17:04 +0200, Andrea Galbusera wrote:
> > Hi Maciej,
> >
> > On Fri, Sep 1, 2017 at 4:08 PM, Maciej Borzęcki <maciej.borzecki@rndi
> > ty.com> wrote:
> > > On Fri, Sep 1, 2017 at 3:54 PM, Andrea Galbusera <gizero@gmail.com>
> > > wrote:
> > > > Hi!
> > > >
> > > > I was trying to share sstate between different hosts, but the
> > > consumer build
> > > > system seems to be unable to use re-use any sstate object. My
> > > scenario is
> > > > setup as follows:
> > > >
> > > > * The cache was populated by a pristine qemux86 core-image-
> > > minimal build of
> > > > morty. This was done in a crops/poky container (running in docker
> > > on Mac)
> > > > * The cache was then served via HTTP
> > >
> > > Make sure that you use a decent HTTP server. Simple `python3 -m
> > > http.server` will quickly choke when the mirror is being checked.
> > > Also
> > > running bitbake -DDD -v makes investigating this much easier.
> >
> > To be honest, the current server was indeed setup with python's
> > SimpleHTTPServer... As you suggest, I checked the verbose debug log
> > and noticed what's happening behind the apparently happy "Checking
> > sstate mirror object availability" step. After a first "SState:
> > Successful fetch test for" that I see correctly served with 200 on
> > the server side, tests for any other sstate object suddenly and
> > systematically fail with logs like this:
> ...
> > DEBUG: checkstatus() urlopen failed: <urlopen error [Errno 9] Bad
> > file descriptor>
>
> More recent bitbake should not fail like that anymore. It's still
> better to use an HTTP server that performs better, though.
>
> commit 6fa07752bbd3ac345cd8617da49a70e0b2dd565f
> Author: Patrick Ohly <patrick.ohly@intel.com>
> Date: Mon Jul 17 15:25:10 2017 +0200
>
> fetch2/wget.py: improve error handling during sstate check
>
> When the sstate is accessed via HTTP, the existence check can fail due
> to network issues, in which case bitbake silently continues without
> sstate.
>
> One such network issue is an HTTP server like Python's own SimpleHTTP
> which closes the TCP connection despite an explicit "Keep-Alive" in
> the HTTP request header. The server does that without a "close" in the
> HTTP response header, so the socket remains in the connection cache,
> leading to "urlopen failed: <urlopen error [Errno 9] Bad file
> descriptor>" (only visible in "bitbake -D -D" output) when trying to
> use the cached connection again.
>
> The connection might also get closed for other reasons (proxy,
> timeouts, etc.), so this is something that the client should be able
> to handle.
>
> This is achieved by checking for the error, removing the bad
> connection, and letting the check_status() method try again with a new
> connection. It is necessary to let the second attempt fail
> permanently, because bad proxy setups have been observed to also lead
> to such broken connections. In that case, we need to abort for real
> after trying twice, otherwise a build would just hang forever.
>
> [YOCTO #11782]
>
> --
> Best Regards, Patrick Ohly
>
> The content of this message is my personal opinion only and although
> I am an employee of Intel, the statements I make here in no way
> represent Intel's position on the issue, nor am I authorized to speak
> on behalf of Intel on this matter.
>
Thank you Patrick for pointing that out. While debugging the issue on morty
I had some reminiscence of seeing your patch on the list, but I wasn't able
to find it back in morty's history hence inferring it landed later on.
Anyway doing this test on morty was a requirement... Thanks again for your
clarification. Would such a patch be suitable for eventually being
back-ported to morty?
[-- Attachment #2: Type: text/html, Size: 5141 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: cannot re-use shared state cache between build hosts
2017-09-04 7:59 ` Andrea Galbusera
@ 2017-09-04 8:56 ` Paul Eggleton
2017-09-07 8:11 ` Andrea Galbusera
0 siblings, 1 reply; 11+ messages in thread
From: Paul Eggleton @ 2017-09-04 8:56 UTC (permalink / raw)
To: Andrea Galbusera; +Cc: yocto
On Monday, 4 September 2017 7:59:55 PM NZST Andrea Galbusera wrote:
> On Mon, Sep 4, 2017 at 9:33 AM, Patrick Ohly <patrick.ohly@intel.com> wrote:
> > More recent bitbake should not fail like that anymore. It's still
> > better to use an HTTP server that performs better, though.
> >
> > commit 6fa07752bbd3ac345cd8617da49a70e0b2dd565f
> > Author: Patrick Ohly <patrick.ohly@intel.com>
> > Date: Mon Jul 17 15:25:10 2017 +0200
> >
> > fetch2/wget.py: improve error handling during sstate check
> >
> > When the sstate is accessed via HTTP, the existence check can fail due
> > to network issues, in which case bitbake silently continues without
> > sstate.
> >
> > One such network issue is an HTTP server like Python's own SimpleHTTP
> > which closes the TCP connection despite an explicit "Keep-Alive" in
> > the HTTP request header. The server does that without a "close" in the
> > HTTP response header, so the socket remains in the connection cache,
> > leading to "urlopen failed: <urlopen error [Errno 9] Bad file
> > descriptor>" (only visible in "bitbake -D -D" output) when trying to
> > use the cached connection again.
> >
> > The connection might also get closed for other reasons (proxy,
> > timeouts, etc.), so this is something that the client should be able
> > to handle.
> >
> > This is achieved by checking for the error, removing the bad
> > connection, and letting the check_status() method try again with a new
> > connection. It is necessary to let the second attempt fail
> > permanently, because bad proxy setups have been observed to also lead
> > to such broken connections. In that case, we need to abort for real
> > after trying twice, otherwise a build would just hang forever.
> >
> > [YOCTO #11782]
> >
>
> Thank you Patrick for pointing that out. While debugging the issue on morty
> I had some reminiscence of seeing your patch on the list, but I wasn't able
> to find it back in morty's history hence inferring it landed later on.
> Anyway doing this test on morty was a requirement... Thanks again for your
> clarification. Would such a patch be suitable for eventually being
> back-ported to morty?
Doesn't look too invasive to me at least, so I'd support it.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: cannot re-use shared state cache between build hosts
2017-09-04 8:56 ` Paul Eggleton
@ 2017-09-07 8:11 ` Andrea Galbusera
0 siblings, 0 replies; 11+ messages in thread
From: Andrea Galbusera @ 2017-09-07 8:11 UTC (permalink / raw)
To: Paul Eggleton; +Cc: yocto@yoctoproject.org
[-- Attachment #1: Type: text/plain, Size: 2948 bytes --]
On Mon, Sep 4, 2017 at 10:56 AM, Paul Eggleton <
paul.eggleton@linux.intel.com> wrote:
> On Monday, 4 September 2017 7:59:55 PM NZST Andrea Galbusera wrote:
> > On Mon, Sep 4, 2017 at 9:33 AM, Patrick Ohly <patrick.ohly@intel.com>
> wrote:
> > > More recent bitbake should not fail like that anymore. It's still
> > > better to use an HTTP server that performs better, though.
> > >
> > > commit 6fa07752bbd3ac345cd8617da49a70e0b2dd565f
> > > Author: Patrick Ohly <patrick.ohly@intel.com>
> > > Date: Mon Jul 17 15:25:10 2017 +0200
> > >
> > > fetch2/wget.py: improve error handling during sstate check
> > >
> > > When the sstate is accessed via HTTP, the existence check can fail
> due
> > > to network issues, in which case bitbake silently continues without
> > > sstate.
> > >
> > > One such network issue is an HTTP server like Python's own
> SimpleHTTP
> > > which closes the TCP connection despite an explicit "Keep-Alive" in
> > > the HTTP request header. The server does that without a "close" in
> the
> > > HTTP response header, so the socket remains in the connection
> cache,
> > > leading to "urlopen failed: <urlopen error [Errno 9] Bad file
> > > descriptor>" (only visible in "bitbake -D -D" output) when trying
> to
> > > use the cached connection again.
> > >
> > > The connection might also get closed for other reasons (proxy,
> > > timeouts, etc.), so this is something that the client should be
> able
> > > to handle.
> > >
> > > This is achieved by checking for the error, removing the bad
> > > connection, and letting the check_status() method try again with a
> new
> > > connection. It is necessary to let the second attempt fail
> > > permanently, because bad proxy setups have been observed to also
> lead
> > > to such broken connections. In that case, we need to abort for real
> > > after trying twice, otherwise a build would just hang forever.
> > >
> > > [YOCTO #11782]
> > >
> >
> > Thank you Patrick for pointing that out. While debugging the issue on
> morty
> > I had some reminiscence of seeing your patch on the list, but I wasn't
> able
> > to find it back in morty's history hence inferring it landed later on.
> > Anyway doing this test on morty was a requirement... Thanks again for
> your
> > clarification. Would such a patch be suitable for eventually being
> > back-ported to morty?
>
> Doesn't look too invasive to me at least, so I'd support it.
>
I did some positive tests with Patrick's patch on top of morty and it
improves a lot the situation even with SimpleHTTPServer. Than I'd like to
propose it is backported to morty and, after further testing, reasonably
should land in pyro too. But I'm unsure how such a request should be
submitted. Since it affects bitbake, what's the proper way and which list
should I send such a request?
[-- Attachment #2: Type: text/html, Size: 4007 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread