Openembedded Core Discussions
 help / color / mirror / Atom feed
* [RFC PATCH 00/30] Add vendor support for go, npm and rust
@ 2025-02-11 15:00 Stefan Herbrechtsmeier
  2025-02-11 15:00 ` [RFC PATCH 01/30] classes: create-spdx-2.2: use expanded FetchData for downloaded packages Stefan Herbrechtsmeier
                   ` (30 more replies)
  0 siblings, 31 replies; 73+ messages in thread
From: Stefan Herbrechtsmeier @ 2025-02-11 15:00 UTC (permalink / raw)
  To: openembedded-core; +Cc: Stefan Herbrechtsmeier, bitbake-devel

From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>

The series adds on-the-fly support for package manager specific
dependencies and vendor directories. It contains the following changes:
1. Adds an early fetch, unpack and patch task to unpack and patch source
   code with an embedded lock file for dependencies.
2. Parse the go.sum, Cargo.lock and package-lock.json lock files and
   resolve the dependencies to SRC_URIs.
3. Save the SRC_URIs in a file and adapt all SRC_URIs users to handle
   the SRC_URI files beside the SRC_URIs in the recipe.
4. Create a package manager specific vendor directory during unpack to
   support additional patching of the dependencies.
5. Add the dependency name and version to the SBOM.
6. Simplify the npm support


Stefan Herbrechtsmeier (30):
  classes: create-spdx-2.2: use expanded FetchData for downloaded
    packages
  lib: spdx30_tasks: use expanded FetchData for download files
  classes: create-spdx-2.2: use name and version for download
    dependencies
  lib: bb: fetch2: add support to unpack .crate files
  lib: oe: add vendor module
  lib: oe: vendor: add cargo support
  lib: oe: vendor: add go support
  lib: oe: vendor: add npm support
  oeqa: oelib: add vendor tests
  conf: bitbake: add SRC_URI_FILES variable
  classes: go: make source directory configurable
  classes: go-mod: make class customizable
  classes: add nodejs-arch class
  classes: base: add get_src_uris and unpack_src_uris functions
  classes: add early fetch, unpack and patch support
  classes: add vendor class
  classes: add vendor class for cargo
  classes: add vendor class for go
  classes: add vendor class for npm
  classes: add vendor_npm_build class
  python3-bcrypt: mirgrate to vendor cargo class
  python3-cryptography: mirgrate to vendor cargo class
  python3-maturin: mirgrate to vendor cargo class
  python3-rpds-py: mirgrate to vendor cargo class
  librsvg: mirgrate to vendor cargo class
  librsvg: update dependecies to fix RUSTSEC-2024-0421
  [DO NOT MERGE] recipes: add crucible go demo
  [DO NOT MERGE] recipes: add node-red npm demo
  [DO NOT MERGE] recipes: add nucleoidai npm demo
  [DO NOT MERGE] classes: spdx: use version 2.2

 bitbake/lib/bb/fetch2/__init__.py             |    2 +-
 .../crucible/crucible2_2023.11.02.bb          |   18 +
 .../node-red/node-red/package-lock.json       | 6096 +++++++++++++++++
 .../node-red/node-red_4.0.8.bb                |   14 +
 .../nucleoidai/nucleoidai_0.7.10.bb           |   11 +
 meta/classes-global/base.bbclass              |   47 +-
 meta/classes-global/patch.bbclass             |   17 +-
 meta/classes-recipe/early.bbclass             |   61 +
 meta/classes-recipe/go-mod.bbclass            |   10 +-
 meta/classes-recipe/go.bbclass                |   22 +-
 meta/classes-recipe/nodejs-arch.bbclass       |   15 +
 meta/classes-recipe/vendor.bbclass            |   28 +
 meta/classes-recipe/vendor_cargo.bbclass      |   46 +
 meta/classes-recipe/vendor_go.bbclass         |   59 +
 meta/classes-recipe/vendor_npm.bbclass        |  115 +
 meta/classes-recipe/vendor_npm_build.bbclass  |   50 +
 meta/classes/archiver.bbclass                 |    4 +-
 meta/classes/buildhistory.bbclass             |    4 +-
 meta/classes/copyleft_compliance.bbclass      |    2 +-
 meta/classes/create-spdx-2.2.bbclass          |   14 +-
 meta/classes/create-spdx.bbclass              |    2 +-
 meta/classes/externalsrc.bbclass              |    2 +-
 meta/conf/bitbake.conf                        |    1 +
 meta/lib/oe/patch.py                          |   10 +-
 meta/lib/oe/spdx30_tasks.py                   |    5 +-
 meta/lib/oe/vendor/__init__.py                |   28 +
 meta/lib/oe/vendor/cargo.py                   |  121 +
 meta/lib/oe/vendor/go.py                      |   96 +
 meta/lib/oe/vendor/npm.py                     |  141 +
 meta/lib/oeqa/selftest/cases/oelib/vendor.py  |  237 +
 .../python/python3-bcrypt-crates.inc          |   84 -
 .../python/python3-bcrypt_4.2.1.bb            |    4 +-
 .../python/python3-cryptography-crates.inc    |   76 -
 .../python/python3-cryptography.bb            |    4 +-
 .../python/python3-maturin-crates.inc         |  712 --
 .../python/python3-maturin_1.8.1.bb           |    4 +-
 .../python/python3-rpds-py-crates.inc         |   54 -
 .../python/python3-rpds-py_0.22.3.bb          |    4 +-
 meta/recipes-gnome/librsvg/librsvg-crates.inc |  590 --
 ...-to-get-an-updated-idna-rustsec-2024.patch |  398 ++
 meta/recipes-gnome/librsvg/librsvg_2.59.2.bb  |    7 +-
 41 files changed, 7633 insertions(+), 1582 deletions(-)
 create mode 100644 meta-selftest/recipes-support/crucible/crucible2_2023.11.02.bb
 create mode 100644 meta-selftest/recipes-support/node-red/node-red/package-lock.json
 create mode 100644 meta-selftest/recipes-support/node-red/node-red_4.0.8.bb
 create mode 100644 meta-selftest/recipes-support/nucleoidai/nucleoidai_0.7.10.bb
 create mode 100644 meta/classes-recipe/early.bbclass
 create mode 100644 meta/classes-recipe/nodejs-arch.bbclass
 create mode 100644 meta/classes-recipe/vendor.bbclass
 create mode 100644 meta/classes-recipe/vendor_cargo.bbclass
 create mode 100644 meta/classes-recipe/vendor_go.bbclass
 create mode 100644 meta/classes-recipe/vendor_npm.bbclass
 create mode 100644 meta/classes-recipe/vendor_npm_build.bbclass
 create mode 100644 meta/lib/oe/vendor/__init__.py
 create mode 100644 meta/lib/oe/vendor/cargo.py
 create mode 100644 meta/lib/oe/vendor/go.py
 create mode 100644 meta/lib/oe/vendor/npm.py
 create mode 100644 meta/lib/oeqa/selftest/cases/oelib/vendor.py
 delete mode 100644 meta/recipes-devtools/python/python3-bcrypt-crates.inc
 delete mode 100644 meta/recipes-devtools/python/python3-cryptography-crates.inc
 delete mode 100644 meta/recipes-devtools/python/python3-maturin-crates.inc
 delete mode 100644 meta/recipes-devtools/python/python3-rpds-py-crates.inc
 delete mode 100644 meta/recipes-gnome/librsvg/librsvg-crates.inc
 create mode 100644 meta/recipes-gnome/librsvg/librsvg/0001-update-url-crate-to-get-an-updated-idna-rustsec-2024.patch

-- 
2.39.5



^ permalink raw reply	[flat|nested] 73+ messages in thread

end of thread, other threads:[~2025-02-13 17:07 UTC | newest]

Thread overview: 73+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-02-11 15:00 [RFC PATCH 00/30] Add vendor support for go, npm and rust Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 01/30] classes: create-spdx-2.2: use expanded FetchData for downloaded packages Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 02/30] lib: spdx30_tasks: use expanded FetchData for download files Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 03/30] classes: create-spdx-2.2: use name and version for download dependencies Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 04/30] lib: bb: fetch2: add support to unpack .crate files Stefan Herbrechtsmeier
2025-02-11 21:22   ` [OE-core] " Richard Purdie
2025-02-11 15:00 ` [RFC PATCH 05/30] lib: oe: add vendor module Stefan Herbrechtsmeier
2025-02-11 21:31   ` [OE-core] " Richard Purdie
2025-02-12  9:27     ` Stefan Herbrechtsmeier
2025-02-12  9:38       ` Richard Purdie
2025-02-12 12:21         ` Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 06/30] lib: oe: vendor: add cargo support Stefan Herbrechtsmeier
2025-02-12 10:32   ` [OE-core] " Alexander Kanavin
2025-02-12 12:45   ` Frédéric Martinsons
2025-02-12 16:29     ` Stefan Herbrechtsmeier
2025-02-12 17:48       ` Frédéric Martinsons
2025-02-13  8:53         ` Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 07/30] lib: oe: vendor: add go support Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 08/30] lib: oe: vendor: add npm support Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 09/30] oeqa: oelib: add vendor tests Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 10/30] conf: bitbake: add SRC_URI_FILES variable Stefan Herbrechtsmeier
2025-02-11 16:22   ` [bitbake-devel] " Peter Kjellerstedt
2025-02-12  8:55     ` Stefan Herbrechtsmeier
2025-02-12  9:49       ` [OE-core] " Alexander Kanavin
     [not found]       ` <18236D0FFBD06B89.28278@lists.openembedded.org>
2025-02-12 10:42         ` Alexander Kanavin
2025-02-11 19:06   ` Peter Kjellerstedt
2025-02-11 15:00 ` [RFC PATCH 11/30] classes: go: make source directory configurable Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 12/30] classes: go-mod: make class customizable Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 13/30] classes: add nodejs-arch class Stefan Herbrechtsmeier
2025-02-12 10:37   ` [OE-core] " Alexander Kanavin
2025-02-11 15:00 ` [RFC PATCH 14/30] classes: base: add get_src_uris and unpack_src_uris functions Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 15/30] classes: add early fetch, unpack and patch support Stefan Herbrechtsmeier
2025-02-11 22:27   ` [OE-core] " Richard Purdie
2025-02-12 12:21     ` Stefan Herbrechtsmeier
2025-02-11 22:32   ` Bruce Ashfield
2025-02-12 12:42     ` Stefan Herbrechtsmeier
2025-02-12 13:55       ` Bruce Ashfield
2025-02-12 14:40         ` Stefan Herbrechtsmeier
2025-02-12 11:08   ` Alexander Kanavin
2025-02-12 16:23     ` Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 16/30] classes: add vendor class Stefan Herbrechtsmeier
2025-02-11 19:17   ` [OE-core] " Peter Kjellerstedt
2025-02-11 15:00 ` [RFC PATCH 17/30] classes: add vendor class for cargo Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 18/30] classes: add vendor class for go Stefan Herbrechtsmeier
2025-02-11 22:59   ` [OE-core] " Bruce Ashfield
2025-02-12 15:23     ` Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 19/30] classes: add vendor class for npm Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 20/30] classes: add vendor_npm_build class Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 21/30] python3-bcrypt: mirgrate to vendor cargo class Stefan Herbrechtsmeier
2025-02-11 21:46   ` [OE-core] " Richard Purdie
2025-02-12 14:36     ` Stefan Herbrechtsmeier
2025-02-12 15:06       ` Richard Purdie
2025-02-12 17:27         ` Stefan Herbrechtsmeier
2025-02-12 15:07       ` Bruce Ashfield
2025-02-12 17:24         ` Stefan Herbrechtsmeier
2025-02-12 17:45           ` Bruce Ashfield
2025-02-12 17:52             ` Richard Purdie
2025-02-13 12:45             ` Stefan Herbrechtsmeier
2025-02-13 17:07               ` Bruce Ashfield
2025-02-11 15:00 ` [RFC PATCH 22/30] python3-cryptography: " Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 23/30] python3-maturin: " Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 24/30] python3-rpds-py: " Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 25/30] librsvg: " Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 26/30] librsvg: update dependecies to fix RUSTSEC-2024-0421 Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 27/30] [DO NOT MERGE] recipes: add crucible go demo Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 28/30] [DO NOT MERGE] recipes: add node-red npm demo Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 29/30] [DO NOT MERGE] recipes: add nucleoidai " Stefan Herbrechtsmeier
2025-02-11 15:00 ` [RFC PATCH 30/30] [DO NOT MERGE] classes: spdx: use version 2.2 Stefan Herbrechtsmeier
2025-02-11 23:14 ` [bitbake-devel] [RFC PATCH 00/30] Add vendor support for go, npm and rust Bruce Ashfield
2025-02-12  8:41   ` Stefan Herbrechtsmeier
2025-02-12 14:11     ` Bruce Ashfield
2025-02-13  8:36       ` Stefan Herbrechtsmeier
2025-02-13 17:01         ` Bruce Ashfield

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox