From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from esa10.bmw.c3s2.iphmx.com (esa10.bmw.c3s2.iphmx.com [68.232.139.98]) by mx.groups.io with SMTP id smtpd.web09.8283.1581944621634547895 for ; Mon, 17 Feb 2020 05:03:44 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@bmw.de header.s=mailing1 header.b=Eu/5jjUG; spf=pass (domain: bmw.de, ip: 68.232.139.98, mailfrom: prvs=3094f87a1=mikko.rapeli@bmw.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bmw.de; i=@bmw.de; q=dns/txt; s=mailing1; t=1581944623; x=1613480623; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=SdkWIR+NwefLZ+HwMXzewVE9ExCMhxaV6L+S4Q8viUU=; b=Eu/5jjUG8TRFnjJKFHi+WEVyWkSbnGCHarrFjTee2XSWtcopcvQlDR/M CgmjB9v7Yz0G8k0s0Vy/hx6gRrELWFZ7M8UwMMqZW1JlkpJ+t9gfPrWbH TJY0t6qeugnK3GV8HK2dB5/vIGIRlsI59oxuCIl5enbB5UjeK8xss8Z9J U=; IronPort-SDR: 756pM6OD2xrVJ8Q0dV7X+qaBDJS9vI1+rNboqAGJHAwzMOlg+fHg4tYfBliAMZKTMGTEhp7rN2 AMUVYitzgdwRps4FT4tldez8LYMCm5TEjjkHReIUEwC1Hs0fYxfgV0MPbV53mwIg0Z9ttVdtLg XU6ZbGqI4Jp4zWf01uACZ5kH68rEPE9ZEWla8H7l20/g5HpPBLR4U35UzUz5MlOGWXHgvcUHmj h/GWlFUv/OYS+NtIQVc3qqVBBvHag6Besi6DiIdR1zy2YvembgzV74V834Bomi5hw16+swudBt gSY= Received: from esagw6.bmwgroup.com (HELO esagw6.muc) ([160.46.252.49]) by esa10.bmw.c3s2.iphmx.com with ESMTP/TLS; 17 Feb 2020 14:03:39 +0100 Received: from esabb1.muc ([160.50.100.31]) by esagw6.muc with ESMTP/TLS; 17 Feb 2020 14:03:38 +0100 Received: from smucm10m.bmwgroup.net (HELO smucm10m.europe.bmw.corp) ([160.48.96.49]) by esabb1.muc with ESMTP/TLS; 17 Feb 2020 14:03:38 +0100 Received: from smucm10k.europe.bmw.corp (160.48.96.47) by smucm10m.europe.bmw.corp (160.48.96.49) with Microsoft SMTP Server (TLS; Mon, 17 Feb 2020 14:03:38 +0100 Received: from smucm10k.europe.bmw.corp ([160.48.96.47]) by smucm10k.europe.bmw.corp ([160.48.96.47]) with mapi id 15.00.1473.005; Mon, 17 Feb 2020 14:03:37 +0100 From: "Mikko Rapeli" To: CC: , Subject: Re: [yocto] Yocto [thud], [zeus] do_fetch and do_unpack failures with offline/online svn build! #yocto #python Thread-Topic: [yocto] Yocto [thud], [zeus] do_fetch and do_unpack failures with offline/online svn build! #yocto #python Thread-Index: AdXcRVMItAsS96VjQ36fsMtuKM+vJgJSsmTA///0WIA= Date: Mon, 17 Feb 2020 13:03:37 +0000 Message-ID: <20200217130337.GS104502@korppu> References: In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 MIME-Version: 1.0 Content-Language: en-US Content-Type: text/plain; charset="iso-8859-1" Content-ID: <4BC64DEAD6B3DA4097A7A061A4E6174C@bmwmail.corp> Content-Transfer-Encoding: quoted-printable On Mon, Feb 17, 2020 at 12:56:13PM +0000, Georgi Georgiev via Lists.Yoctopr= oject.Org wrote: > Another serious issue with svn fetcher, always occurring when doing incr= emental builds. I am putting it here because I think both issues are relate= d. Please suggest what to do. The error we get is shown below. This time it= happens always. The only workaround is to clean and build. As soon as we m= ake modification of the recipe (PV change only) it fails: >=20 > ERROR: ww-pacman-121204-r0 do_fetch: Error executing a python function i= n exec_python_func() autogenerated: >=20 > The stack trace of python calls that resulted in this exception/failure = was: > File: 'exec_python_func() autogenerated', lineno: 2, function: > 0001: > *** 0002:base_do_fetch(d) > 0003: > File: '/data/home/w23698/projects/proj/build/../sources/poky/meta/classe= s/base.bbclass', lineno: 163, function: base_do_fetch > 0159: return > 0160: > 0161: try: > 0162: fetcher =3D bb.fetch2.Fetch(src_uri, d) > *** 0163: fetcher.download() > 0164: except bb.fetch2.BBFetchException as e: > 0165: bb.fatal(str(e)) > 0166:} > 0167: > File: '/data/home/w23698/projects/proj/sources/poky/bitbake/lib/bb/fetch= 2/__init__.py', lineno: 1678, function: download > 1674: try: > 1675: if not trusted_network(self.d, ud.url)= : > 1676: raise UntrustedUrl(ud.url) > 1677: logger.debug(1, "Trying Upstream") > *** 1678: m.download(ud, self.d) > 1679: if hasattr(m, "build_mirror_data"): > 1680: m.build_mirror_data(ud, self.d) > 1681: localpath =3D ud.localpath > 1682: # early checksum verify, so that if ch= ecksum mismatched, > File: '/data/home/w23698/projects/proj/sources/poky/bitbake/lib/bb/fetch= 2/svn.py', lineno: 150, function: download > 0146: if not ("externals" in ud.parm and ud.parm["extern= als"] =3D=3D "nowarn"): > 0147: # Warn the user if this had externals (won't c= atch them all) > 0148: output =3D runfetchcmd("svn propget svn:extern= als || true", d, workdir=3Dud.moddir) > 0149: if output: > *** 0150: if "--ignore-externals" in svnfetchcmd.spl= it(): > 0151: bb.warn("%s contains svn:externals." %= ud.url) > 0152: bb.warn("These should be added to the = recipe SRC_URI as necessary.") > 0153: bb.warn("svn fetch has ignored externa= ls:\n%s" % output) > 0154: bb.warn("To disable this warning add '= ;externals=3Dnowarn' to the url.") > Exception: UnboundLocalError: local variable 'svnfetchcmd' referenced be= fore assignment >=20 > ERROR: Logfile of failure stored in: /data/home/w23698/projects/proj/bui= ld/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121204-r0/temp/log= .do_fetch.18853 Hmm. Rings a bell. I run builds with a local non-upstreamed patch: Author: Mikko Rapeli AuthorDate: Fri Sep 6 14:15:20 2019 +0200 Commit: Mikko Rapeli CommitDate: Fri Sep 6 14:25:15 2019 +0200 svn fetcher: allow "svn propget svn:externals" to fail Not all servers and repositories have this property set which results in failures like this when actual svn checkout command succeeded: svn: warning: W200017: Property 'svn:externals' not found on '' svn: E200000: A problem occurred; see other errors for details Signed-off-by: Mikko Rapeli --- a/bitbake/lib/bb/fetch2/svn.py +++ b/bitbake/lib/bb/fetch2/svn.py @@ -145,7 +145,7 @@ class Svn(FetchMethod): =20 if not ("externals" in ud.parm and ud.parm["externals"] = =3D=3D "nowarn"): # Warn the user if this had externals (won't catch th= em all) - output =3D runfetchcmd("svn propget svn:externals", d= , workdir=3Dud.moddir) + output =3D runfetchcmd("svn propget svn:externals || = true", d, workdir=3Dud.moddir) if output: if "--ignore-externals" in svnfetchcmd.split(): bb.warn("%s contains svn:externals." % ud.url= ) I think this would help. Cheers, -Mikko > -----Original Message----- > From: Georgi Georgiev=20 > Sent: Wednesday, February 05, 2020 6:57 PM > To: Georgi Georgiev ; Khem Raj ; yocto@lists.yoctoproject.org > Subject: Re: [yocto] Yocto [thud], [zeus] do_fetch and do_unpack failure= s with offline/online svn build! #yocto #python >=20 > Sorry Khem, > With esc character '\' before & it can't take the full path. So briefly: >=20 > Yocto build with char '\' before & in SRC_URI: > =09online - OK > =09offline - Error - svn: E170013: Unable to connect to a repository... > Yocto build without '\' in SRC_URI: > =09online and offline same error - do_unpack: Unpack failure for URL - "= package".tar.gz is present in DL_DIR >=20 > Thanks >=20 >=20 >=20 > -----Original Message----- > From: yocto@lists.yoctoproject.org [mailto:yocto@lists.yoctoproject.org]= On Behalf Of Georgi Georgiev via Lists.Yoctoproject.Org > Sent: Wednesday, February 05, 2020 5:26 PM > To: Khem Raj ; yocto@lists.yoctoproject.org > Cc: yocto@lists.yoctoproject.org > Subject: Re: [EXTERNAL] Re: [yocto] Yocto [thud], [zeus] do_fetch and do= _unpack failures with offline/online svn build! #yocto #python >=20 > Hi Khem, >=20 > Yes, no issues with that. When I am connected to network it fetches the = code with and without escape character before & in the path. > svn co " https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A__cocosubve= rsion_svn_Embedded_Valve-26Actuator_DVPII_trunk_SOCPACManEnvEngKeys&d=3DDwI= GaQ&c=3Dy6L7g950KfMp92YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=3DkHtJrQGzfH1ZmfsNkJp= YuH-jtNpv8yMDkqAmsRP99mc&m=3DvXia-xlcxuLr2fSmBogZpMIaOntXLKWv3mkxcRgtUc4&s= =3DUYH9lueDIeMGZndMxgbZgb76A5EfqN-MR58DBv1iriI&e=3D " works fine and fetch= es the package.=20 >=20 > Georgi >=20 > -----Original Message----- > From: Khem Raj [mailto:raj.khem@gmail.com] > Sent: Friday, January 31, 2020 8:33 PM > To: Georgi Georgiev ; yocto@lists.yoctopro= ject.org > Subject: [EXTERNAL] Re: [yocto] Yocto [thud], [zeus] do_fetch and do_unp= ack failures with offline/online svn build! #yocto #python >=20 > On 1/31/20 4:02 AM, georgi.georgiev via Lists.Yoctoproject.Org wrote: > > Hello Community, > > This is the third time I am asking for support on this issue. This=20 > > time I decided to use the web form. > > In our project we have a requirement to be able to build the project= =20 > > offline. E.g. on the field without any network connection. When we are= = =20 > > connected with the recipe mentioned below we don't have issues: > >=20 >=20 >=20 > svn: E170013: Unable to connect to a repository at URL 'https://urldefen= se.proofpoint.com/v2/url?u=3Dhttp-3A__cocosubversion_svn_Embedded_Valve-26A= ctuator_DVPII_trunk_SOCPACManEnvEngKeys_trunk&d=3DDwIGaQ&c=3Dy6L7g950KfMp92= YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=3DkHtJrQGzfH1ZmfsNkJpYuH-jtNpv8yMDkqAmsRP99= mc&m=3Docx6BoVSrDQGWUvUUVvgKtuJbT7eH7jFSjCy1Ys73Vw&s=3DOTunnmsi_tCUt3JytmU2= Hs7i7Xnqhl8-2CMsqmzIl90&e=3D ' > svn: E670003: Temporary failure in name resolution >=20 > can you try checking out the repo out side of fetcher and see if the mac= hine can fetch the url you are using in SRC_URI ? >=20 > > *SUMMARY =3D "PACMan - Parameter And Configuration MANager"* *LICENSE = = =3D > > "CLOSED"* *inherit systemd useradd* *REQUIRED_DISTRO_FEATURES =3D > > "systemd"* *# SVN revision* *PV =3D "121026"* *# Name of SVN project* > > *PACMAN_PROJ_NAME=3D"SOCPACManEnvEngKeys"* > > *SRC_URI =3D > > "svn://cocosubversion/svn/Embedded/Valve%5C&Actuator/DVPII/trunk/$%7BP > > ACMAN_PROJ_NAME%7D;module=3Dtrunk;protocol=3Dhttp;externals=3Dallowed;= rev=3D$% > > 7BPV%7D%22* *SRC_URI +=3D "file://ww-authpacman.service%22* *SRC_URI += = =3D > > "file://ww-pacman.service%22* *S =3D "${WORKDIR}/trunk/"* *# ${PN}-sys= :=20 > > content related to system, which goes to base rootfs (only .service=20 > > file and symlinks)* *# ${PN}:=A0=A0=A0=A0=A0 real content which may go= to=20 > > separate partition* *PACKAGES =3D+ " ${PN}-sys"* ......... > > When disconnect the network, erase sstate-cache, cache and tmp I see= =20 > > log file attached (log.do_fetch.32757) and the following output: > >=20 > > *ERROR: ww-pacman-121026-r0 do_fetch: Fetcher failure: Fetch command= =20 > > export PSEUDO_DISABLED=3D1; unset _PYTHON_SYSCONFIGDATA_NAME; export= =20 > > DBUS_SESSION_BUS_ADDRESS=3D"unix:path=3D/run/user/1000/bus"; export=20 > > SSH_AGENT_PID=3D"11412"; export=20 > > SSH_AUTH_SOCK=3D"/run/user/1000/keyring/ssh"; export=20 > > PATH=3D"/home/w23698/projects/proj_dvp2/build_dvp2/tmp/sysroots-uninat= iv > > e/x86_64-linux/usr/bin:/home/w23698/projects/proj_dvp2/sources/poky/sc > > ripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-n > > eon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/b > > in/arm-poky-linux-gnueabi:/home/w23698/projects/proj_dvp2/build_dvp2/t > > mp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe- > > sysroot/usr/bin/crossscripts:/home/w23698/projects/proj_dvp2/build_dvp > > 2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/reci > > pe-sysroot-native/usr/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/ > > tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe > > -sysroot-native/usr/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp > > /work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sy > > sroot-native/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/ > > cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot- > > native/bin:/home/w23698/projects/proj_dvp2/sources/poky/bitbake/bin:/h > > ome/w23698/projects/proj_dvp2/build_dvp2/tmp/hosttools"; > > export HOME=3D"/home/w23698"; /usr/bin/env svn --non-interactive=20 > > --trust-server-cert update --no-auth-cache -r 121026 failed with exit= =20 > > code 1, output:* *Updating '.':* > > *svn: E170013: Unable to connect to a repository at URL=20 > > 'https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A__cocosubversion_= sv > > n_Embedded_Valve-26Actuator_DVPII_trunk_SOCPACManEnvEngKeys_trunk-27-2 > > A&d=3DDwIGaQ&c=3Dy6L7g950KfMp92YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=3DkHtJr= QGzfH1 > > ZmfsNkJpYuH-jtNpv8yMDkqAmsRP99mc&m=3Docx6BoVSrDQGWUvUUVvgKtuJbT7eH7jFS= jC > > y1Ys73Vw&s=3DFfv1JU1QYBh4g49fmoLnsDSFgMMBc_5MbOpy59QUS18&e=3D > > *svn: E670003: Temporary failure in name resolution* > > *ERROR: ww-pacman-121026-r0 do_fetch: Fetcher failure for URL:=20 > > 'svn://cocosubversion/svn/Embedded/Valve%5C&Actuator/DVPII/trunk/SOCPA= CManEnvEngKeys;module=3Dtrunk;protocol=3Dhttp;externals=3Dallowed;rev=3D121= 026'.=20 > > Unable to fetch URL from any source.* > > *ERROR: Logfile of failure stored in:=20 > > /home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-po > > ky-linux-gnueabi/ww-pacman/121026-r0/temp/log.do_fetch.32757* > > *ERROR: Task > > (/home/w23698/projects/proj_dvp2/build_dvp2/../sources/meta-ww-dvp2/re > > cipes-ww/ww-pacman/ww-pacman.bb:do_fetch) > > failed with exit code '1'* > >=20 > > When remove the '\' character in SRC_URI, e.g. to become: > >=20 > > *SRC_URI =3D > > "svn://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/$%7BPACM > > AN_PROJ_NAME%7D;module=3Dtrunk;protocol=3Dhttp;externals=3Dallowed;rev= = =3D$%7BP > > V%7D%22* > >=20 > > In connected and not connected to network do_fetch() passes=20 > > successfully but I see one and same error (log.do_unpack.25226) output= : > >=20 > > *ERROR: ww-pacman-121026-r0 do_unpack: Unpack failure for URL:=20 > > 'svn://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/SOCPACMa= nEnvEngKeys;module=3Dtrunk;protocol=3Dhttp;externals=3Dallowed;rev=3D121026= '.=20 > > Unpack command > > PATH=3D"/home/w23698/projects/proj_dvp2/build_dvp2/tmp/sysroots-uninat= ive/x86_64-linux/usr/bin:/home/w23698/projects/proj_dvp2/sources/poky/scrip= ts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky= -linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin/arm-poky-l= inux-gnueabi:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf= -neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot/usr/bin/crosssc= ripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-p= oky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/sbin:/home/= w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gn= ueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin:/home/w23698/projec= ts/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacm= an/121026-r0/recipe-sysroot-native/sbin:/home/w23698/projects/proj_dvp2/bui= ld_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/rec= ipe-sysroot-native/bin:/home/w23698/projects/proj_dvp2/sources/poky/bitbake= /bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/hosttools"=20 > > tar xz --no-same-owner -f > > /home/w23698/projects/proj_dvp2/build_dvp2/downloads/trunk_cocosubvers > > ion_.svn.Embedded.Valve&Actuator.DVPII.trunk.SOCPACManEnvEngKeys_12102 > > 6_.tar.gz > > failed with return value 127* > > *ERROR: Logfile of failure stored in:=20 > > /home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-po > > ky-linux-gnueabi/ww-pacman/121026-r0/temp/log.do_unpack.25226* > > *ERROR: Task > > (/home/w23698/projects/proj_dvp2/build_dvp2/../sources/meta-ww-dvp2/re > > cipes-ww/ww-pacman/ww-pacman.bb:do_unpack) > > failed with exit code '1' > > * > > I don't it this matters but, the build machine is baremetal Ubuntu > > 18.04.3 LTS. In all cases the packed tar.gz remain in downloads=20 > > directory with one name! > >=20 > > Cordially, > > Georgi > >=20 > >=20 > >=20 > >=20 >=20 >=20 > *** > The information in this email is confidential and intended solely for th= e individual or entity to whom it is addressed.=A0=A0If you have received t= his email in error please notify the sender by return e-mail, delete this e= mail, and refrain from any disclosure or action based on the information. > *** >=20 > *** > The information in this email is confidential and intended solely for th= e individual or entity to whom it is addressed.=A0=A0If you have received t= his email in error please notify the sender by return e-mail, delete this e= mail, and refrain from any disclosure or action based on the information. > *** >=20