From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26458C0015E for ; Tue, 25 Jul 2023 09:30:53 +0000 (UTC) Subject: Re: [PATCH] fetch2:init check if path is none before calculating checksums To: bitbake-devel@lists.openembedded.org From: belouargamohamed@gmail.com X-Originating-Location: Salon-de-Provence, Provence-Alpes-Côte d'Azur, FR (193.253.180.72) X-Originating-Platform: Linux Firefox 114 User-Agent: GROUPS.IO Web Poster MIME-Version: 1.0 Date: Tue, 25 Jul 2023 02:30:47 -0700 References: In-Reply-To: Message-ID: <3920.1690277447202288940@lists.openembedded.org> Content-Type: multipart/alternative; boundary="KczK1bsOnlOWHpTQNRoL" List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 25 Jul 2023 09:30:53 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14886 --KczK1bsOnlOWHpTQNRoL Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The error happens when I was testing if npm works better. I just generated a recipe and a shrinkwrap using the command: - devtool add "npm://registry.npmjs.org;package=3Dexpress-ws;version=3D5.0.= 2;" Then I built it with: - bitbake express-ws And this error happened: ERROR: express-ws-5.0.2-r0 do_create_spdx: Error executing a python functio= n in exec_func_python() autogenerated: The stack trace of python calls that resulted in this exception/failure was= : File: 'exec_func_python() autogenerated', lineno: 2, function: 0001: *** 0002:do_create_spdx(d) 0003: File: '/home/twigabecomp/yocto/literal/debug/poky/meta/classes/create-spdx-= 2.2.bbclass', lineno: 598, function: do_create_spdx 0594: 0595:=C2=A0=C2=A0=C2=A0 doc.packages.append(recipe) 0596:=C2=A0=C2=A0=C2=A0 doc.add_relationship(doc, "DESCRIBES", recipe) 0597: *** 0598:=C2=A0=C2=A0=C2=A0 add_download_packages(d, doc, recipe) 0599: 0600:=C2=A0=C2=A0=C2=A0 if process_sources(d) and include_sources: 0601:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 recipe_archive =3D deploy_d= ir_spdx / "recipes" / (doc.name + ".tar.zst") 0602:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 with optional_tarfile(recip= e_archive, archive_sources) as archive: File: '/home/twigabecomp/yocto/literal/debug/poky/meta/classes/create-spdx-= 2.2.bbclass', lineno: 438, function: add_download_packages 0434: 0435:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if = f.method.supports_srcrev(): 0436:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 uri =3D uri + "@" + f.revisions[name] 0437: *** 0438:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= if f.method.supports_checksum(f): 0439:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 for checksum_id in CHECKSUM_LIST: 0440:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if checksum_id.upper() not in= oe.spdx.SPDXPackage.ALLOWED_CHECKSUMS: 0441:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 conti= nue 0442: File: '/home/twigabecomp/yocto/literal/debug/poky/bitbake/lib/bb/fetch2/__i= nit__.py', lineno: 1406, function: supports_checksum 1402:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Is localpath something that= can be represented by a checksum? 1403:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 """ 1404: 1405:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # We cannot compute checksu= ms for directories *** 1406:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if os.path.isdir(urldat= a.localpath): 1407:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ret= urn False 1408:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return True 1409: 1410:=C2=A0=C2=A0=C2=A0 def recommends_checksum(self, urldata): File: '/usr/lib/python3.10/genericpath.py', lineno: 42, function: isdir 0038:# can be true for the same path on systems that support symlinks 0039:def isdir(s): 0040:=C2=A0=C2=A0=C2=A0 """Return true if the pathname refers to an existin= g directory.""" 0041:=C2=A0=C2=A0=C2=A0 try: *** 0042:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 st =3D os.stat(s) 0043:=C2=A0=C2=A0=C2=A0 except (OSError, ValueError): 0044:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return False 0045:=C2=A0=C2=A0=C2=A0 return stat.S_ISDIR(st.st_mode) 0046: Exception: TypeError: stat: path should be string, bytes, os.PathLike or in= teger, not NoneType ERROR: Logfile of failure stored in: /home/twigabecomp/yocto/literal/debug/= build/tmp/work/core2-64-poky-linux/express-ws/5.0.2-r0/temp/log.do_create_s= pdx.3703234 ERROR: Task (/home/twigabecomp/yocto/literal/debug/build/workspace/recipes/= express-ws/express-ws_5.0.2.bb:do_create_spdx) failed with exit code '1' Knowing that urldata =3D {'donestamp': None, 'needdonestamp': False, 'local= file': '', 'localpath': None, 'lockfile': None, 'mirrortarballs': [], 'base= name': None, 'basepath': None, 'type': 'npmsw', 'host': '', 'path': '/home/= twigabecomp/yocto/literal/debug/build/workspace/recipes/express-ws/express-= ws/npm-shrinkwrap.json', 'user': '', 'pswd': '', 'parm': OrderedDict(), 'da= te': '20230725', 'url': 'npmsw:///home/twigabecomp/yocto/literal/debug/buil= d/workspace/recipes/express-ws/express-ws/npm-shrinkwrap.json', 'setup': Fa= lse, 'names': ['default'], 'method': , 'shrinkwrap_file': '/home/twigabecomp/yocto/literal/deb= ug/build/workspace/recipes/express-ws/express-ws/npm-shrinkwrap.json', 'dev= ': False, 'deps': [{'url': 'https://registry.npmjs.org/ws/-/ws-7.5.9.tgz;do= wnloadfilename=3Dnpm2/ws-7.5.9.tgz;sha512sum=3D17e3fd26297b52248a4a4a692220= fde1d374ec0c2f162c6f2c88f53a0d5197d18632e362a499d5f49ce30fa5eeaa601e8acc06b= d498d2e3af9705b97c0d4bbed', 'localpath': '/home/twigabecomp/yocto/literal/d= ebug/build/downloads/npm2/ws-7.5.9.tgz', 'extrapaths': ['/home/twigabecomp/= yocto/literal/debug/build/downloads/npm2/ws-7.5.9.tgz.resolved'], 'destsuff= ix': 'node_modules/ws', 'unpack': True}], 'proxy': , 'md5_name': 'md5sum', 'md5_expected': None, 'sha256_nam= e': 'sha256sum', 'sha256_expected': None, 'sha1_name': 'sha1sum', 'sha1_exp= ected': None, 'sha384_name': 'sha384sum', 'sha384_expected': None, 'sha512_= name': 'sha512sum', 'sha512_expected': None, 'ignore_checksums': False} I think this is a known issue of how npm dependencies are represented, beca= use the real localpath(s) is urldata.deps[i].localpath Mohamed BELOUARGA Thanks --KczK1bsOnlOWHpTQNRoL Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable The error happens when I was testing if npm works better.

I just= generated a recipe and a shrinkwrap using the command:
- devtool add = "npm://registry.npmjs.org;package=3Dexpress-ws;version=3D5.0.2;"
Then = I built it with:
- bitbake express-ws
And this error happened:ERROR: express-ws-5.0.2-r0 do_create_spdx: Error executing a python func= tion in exec_func_python() autogenerated:

The stack trace of pyt= hon calls that resulted in this exception/failure was:
File: 'exec_fun= c_python() autogenerated', lineno: 2, function: <module>
 &= nbsp;   0001:
 *** 0002:do_create_spdx(d)
 &n= bsp;   0003:
File: '/home/twigabecomp/yocto/literal/debug/po= ky/meta/classes/create-spdx-2.2.bbclass', lineno: 598, function: do_create_= spdx
     0594:
     0595= :    doc.packages.append(recipe)
   &nbs= p; 0596:    doc.add_relationship(doc, "DESCRIBES", recipe)     0597:
 *** 0598:    a= dd_download_packages(d, doc, recipe)
     0599:     0600:    if process_sources(d) a= nd include_sources:
     0601:   &n= bsp;    recipe_archive =3D deploy_dir_spdx / "recipes" / (do= c.name + ".tar.zst")
     0602:   &= nbsp;    with optional_tarfile(recipe_archive, archive_sourc= es) as archive:
File: '/home/twigabecomp/yocto/literal/debug/poky/meta= /classes/create-spdx-2.2.bbclass', lineno: 438, function: add_download_pack= ages
     0434:
     0435= :            if f.me= thod.supports_srcrev():
     0436:  &nbs= p;             = uri =3D uri + "@" + f.revisions[name]
     0437: *** 0438:         &n= bsp;  if f.method.supports_checksum(f):
     = 0439:           &nbs= p;    for checksum_id in CHECKSUM_LIST:
  &nb= sp;  0440:          =           if checksum_id.upper= () not in oe.spdx.SPDXPackage.ALLOWED_CHECKSUMS:
   &nb= sp; 0441:           =              co= ntinue
     0442:
File: '/home/twigabecomp/yo= cto/literal/debug/poky/bitbake/lib/bb/fetch2/__init__.py', lineno: 1406, fu= nction: supports_checksum
     1402:  &n= bsp;     Is localpath something that can be represented= by a checksum?
     1403:    =     """
     1404:
  = ;   1405:        # We cannot c= ompute checksums for directories
 *** 1406:   &nbs= p;    if os.path.isdir(urldata.localpath):
  =    1407:         &nb= sp;  return False
     1408:   = ;     return True
     1409:     1410:    def recommends_checksu= m(self, urldata):
File: '/usr/lib/python3.10/genericpath.py', lineno: = 42, function: isdir
     0038:# can be true for th= e same path on systems that support symlinks
     = 0039:def isdir(s):
     0040:    ""= "Return true if the pathname refers to an existing directory."""
 = ;    0041:    try:
 *** 0042: =        st =3D os.stat(s)
  &nb= sp;  0043:    except (OSError, ValueError):
 =     0044:        return F= alse
     0045:    return stat.S_IS= DIR(st.st_mode)
     0046:
Exception: TypeErr= or: stat: path should be string, bytes, os.PathLike or integer, not NoneTyp= e

ERROR: Logfile of failure stored in: /home/twigabecomp/yocto/l= iteral/debug/build/tmp/work/core2-64-poky-linux/express-ws/5.0.2-r0/temp/lo= g.do_create_spdx.3703234
ERROR: Task (/home/twigabecomp/yocto/literal/= debug/build/workspace/recipes/express-ws/express-ws_5.0.2.bb:do_create_spdx= ) failed with exit code '1'


Knowing that urldata =3D {'don= estamp': None, 'needdonestamp': False, 'localfile': '', 'localpath': None, = 'lockfile': None, 'mirrortarballs': [], 'basename': None, 'basepath': None,= 'type': 'npmsw', 'host': '', 'path': '/home/twigabecomp/yocto/literal/debu= g/build/workspace/recipes/express-ws/express-ws/npm-shrinkwrap.json', 'user= ': '', 'pswd': '', 'parm': OrderedDict(), 'date': '20230725', 'url': 'npmsw= :///home/twigabecomp/yocto/literal/debug/build/workspace/recipes/express-ws= /express-ws/npm-shrinkwrap.json', 'setup': False, 'names': ['default'], 'me= thod': <bb.fetch2.npmsw.NpmShrinkWrap object at 0x7f50d8c9d600>, 'shr= inkwrap_file': '/home/twigabecomp/yocto/literal/debug/build/workspace/recip= es/express-ws/express-ws/npm-shrinkwrap.json', 'dev': False, 'deps': [{'url= ': 'https://registry.npmjs.org/ws/-/ws-7.5.9.tgz;downloadfilename=3Dnpm2/ws= -7.5.9.tgz;sha512sum=3D17e3fd26297b52248a4a4a692220fde1d374ec0c2f162c6f2c88= f53a0d5197d18632e362a499d5f49ce30fa5eeaa601e8acc06bd498d2e3af9705b97c0d4bbe= d', 'localpath': '/home/twigabecomp/yocto/literal/debug/build/downloads/npm= 2/ws-7.5.9.tgz', 'extrapaths': ['/home/twigabecomp/yocto/literal/debug/buil= d/downloads/npm2/ws-7.5.9.tgz.resolved'], 'destsuffix': 'node_modules/ws', = 'unpack': True}], 'proxy': <bb.fetch2.Fetch object at 0x7f50d30ff250>= , 'md5_name': 'md5sum', 'md5_expected': None, 'sha256_name': 'sha256sum', '= sha256_expected': None, 'sha1_name': 'sha1sum', 'sha1_expected': None, 'sha= 384_name': 'sha384sum', 'sha384_expected': None, 'sha512_name': 'sha512sum'= , 'sha512_expected': None, 'ignore_checksums': False}

I think th= is is a known issue of how npm dependencies are represented, because the re= al localpath(s) is urldata.deps[i].localpath

Mohamed BELOUARGAThanks --KczK1bsOnlOWHpTQNRoL--