From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 475EDC4167B for ; Tue, 14 Nov 2023 16:50:03 +0000 (UTC) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by mx.groups.io with SMTP id smtpd.web10.17589.1699980595776904724 for ; Tue, 14 Nov 2023 08:49:57 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=ZQh5hnLX; spf=pass (domain: bootlin.com, ip: 217.70.183.198, mailfrom: michael.opdenacker@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id 72851C0002; Tue, 14 Nov 2023 16:49:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1699980593; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ayMQUmwitYxcSbUUdomSwty1Pw66ZCj/NCq0nCet32U=; b=ZQh5hnLXvSJ7nxcQ16mXUS2KGaZ0Lp+dw6j5QF7X757uBRXjqtIzBU7HQDd65NtbCWcmeP DEg23zEeCB4I/sLZ/8ClpHk3FXmzYaSk9TnLnye9RGvkuTp/lvGefd30m9u9CcMwArJM37 NWVkAVgqKDgS3KStv6o8PIgEP9DeWCiBIyWr7qdDlhzskOUomzLrkHb/YD+tLTlwL1Sva5 AA/pzFADrmb49/hlDQSfWao5bq/5cfpj8dQIGFCbIDTGXFkchsK8bd16KVk8jda79ir5xv H5aKZHmANLMhJarWv7uz1Nlxn8MZhwJHbjTbT6AJ9VaOCvoqTOSOcsjH0bvJBA== From: michael.opdenacker@bootlin.com To: docs@lists.yoctoproject.org Cc: Michael Opdenacker , Talel BELHAJSALEM Subject: [kirkstone][PATCH 4/7] overview-manual: concepts: Add Bitbake Tasks Map Date: Tue, 14 Nov 2023 17:49:39 +0100 Message-Id: <20231114164942.133472-5-michael.opdenacker@bootlin.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231114164942.133472-1-michael.opdenacker@bootlin.com> References: <20231114164942.133472-1-michael.opdenacker@bootlin.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-GND-Sasl: michael.opdenacker@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 14 Nov 2023 16:50:03 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/4624 From: Michael Opdenacker From: BELHADJ SALEM Talel Create a Map to detail how BitBake handles a recipe's tasks and its compile/runtime dependencies along with detailed comments. Signed-off-by: Talel BELHAJSALEM Reviewed-by: Michael Opdenacker --- documentation/overview-manual/concepts.rst | 170 ++++++++++++++++++ .../overview-manual/svg/bitbake_tasks_map.svg | 4 + 2 files changed, 174 insertions(+) create mode 100644 documentation/overview-manual/svg/bitbake_tasks_map.svg diff --git a/documentation/overview-manual/concepts.rst b/documentation/ove= rview-manual/concepts.rst index 76e02eafff..4a8ea0f611 100644 --- a/documentation/overview-manual/concepts.rst +++ b/documentation/overview-manual/concepts.rst @@ -2230,3 +2230,173 @@ For more information, see the BitBake User Manual. You can also reference the "`Why Not Fakeroot? `__" article for background information on Fakeroot and Pseudo. + +BitBake Tasks Map +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +To understand how BitBake operates in the build directory and environment +we can consider the following recipes and diagram, to have full picture +about the tasks that BitBake runs to generate the final package file +for the recipe. + +We will have two recipes as an example: + +- ``libhello``: A recipe that provides a shared library +- ``sayhello``: A recipe that uses ``libhello`` library to do its job + +.. note:: + + ``sayhello`` depends on ``libhello`` at compile time as it needs the sh= ared + library to do the dynamic linking process. It also depends on it at run= time + as the shared library loader needs to find the library. + For more details about dependencies check :ref:`ref-varlocality-recipe-= dependencies`. + +``libhello`` sources are as follows: + +- ``LICENSE``: This is the license associated with this library +- ``Makefile``: The file used by ``make`` to build the library +- ``hellolib.c``: The implementation of the library +- ``hellolib.h``: The C header of the library + +``sayhello`` sources are as follows: + +- ``LICENSE``: This is the license associated with this project +- ``Makefile``: The file used by ``make`` to build the project +- ``sayhello.c``: The source file of the project + +Before presenting the contents of each file, here are the steps +that we need to follow to accomplish what we want in the first place, +which is integrating ``sayhello`` in our root file system: + +#. Create a Git repository for each project with the corresponding files + +#. Create a recipe for each project + +#. Make sure that ``sayhello`` recipe :term:`DEPENDS` on ``libhello`` + +#. Make sure that ``sayhello`` recipe :term:`RDEPENDS` on ``libhello`` + +#. Add ``sayhello`` to :term:`IMAGE_INSTALL` to integrate it into + the root file system + +The following are the contents of ``libhello/Makefile``:: + + LIB=3Dlibhello.so + + all: $(LIB) + + $(LIB): hellolib.o + $(CC) $< -Wl,-soname,$(LIB).1 -fPIC $(LDFLAGS) -shared -o $(LIB).1.0 + + %.o: %.c + $(CC) -c $< + + clean: + rm -rf *.o *.so* + +.. note:: + + When creating shared libraries, it is strongly recommended to follow th= e Linux + conventions and guidelines (see `this article + `__ + for some background). + +.. note:: + + When creating ``Makefile`` files, it is strongly recommended to use ``C= C``, ``LDFLAGS`` + and ``CFLAGS`` as BitBake will set them as environment variables accord= ing + to your build configuration. + +The following are the contents of ``libhello/hellolib.h``:: + + #ifndef HELLOLIB_H + #define HELLOLIB_H + + void Hello(); + + #endif + +The following are the contents of ``libhello/hellolib.c``:: + + #include + + void Hello(){ + puts("Hello from a Yocto demo \n"); + } + +The following are the contents of ``sayhello/Makefile``:: + + EXEC=3Dsayhello + LDFLAGS +=3D -lhello + + all: $(EXEC) + + $(EXEC): sayhello.c + $(CC) $< $(LDFLAGS) $(CFLAGS) -o $(EXEC) + + clean: + rm -rf $(EXEC) *.o + +The following are the contents of ``sayhello/sayhello.c``:: + + #include + + int main(){ + Hello(); + return 0; + } + +The following are the contents of ``libhello_0.1.bb``:: + + SUMMARY =3D "Hello demo library" + DESCRIPTION =3D "Hello shared library used in Yocto demo" + + # NOTE: Set the License according to the LICENSE file of your project + # and then add LIC_FILES_CHKSUM accordingly + LICENSE =3D "CLOSED" + + # Assuming the branch is main + # Change accordingly + SRC_URI =3D "git://github.com//libhello;branch=3Dmain;protoco= l=3Dhttps" + + S =3D "${WORKDIR}/git" + + do_install(){ + install -d ${D}${includedir} + install -d ${D}${libdir} + + install hellolib.h ${D}${includedir} + oe_soinstall ${PN}.so.${PV} ${D}${libdir} + } + +The following are the contents of ``sayhello_0.1.bb``:: + + SUMMARY =3D "SayHello demo" + DESCRIPTION =3D "SayHello project used in Yocto demo" + + # NOTE: Set the License according to the LICENSE file of your project + # and then add LIC_FILES_CHKSUM accordingly + LICENSE =3D "CLOSED" + + # Assuming the branch is main + # Change accordingly + SRC_URI =3D "git://github.com//sayhello;branch=3Dmain;protoco= l=3Dhttps" + + DEPENDS +=3D "libhello" + RDEPENDS:${PN} +=3D "libhello" + + S =3D "${WORKDIR}/git" + + do_install(){ + install -d ${D}/usr/bin + install -m 0700 sayhello ${D}/usr/bin + } + +After placing the recipes in a custom layer we can run ``bitbake sayhello`` +to build the recipe. + +The following diagram shows the sequences of tasks that BitBake +executes to accomplish that. + +.. image:: svg/bitbake_tasks_map.* + :width: 100% diff --git a/documentation/overview-manual/svg/bitbake_tasks_map.svg b/docu= mentation/overview-manual/svg/bitbake_tasks_map.svg new file mode 100644 index 0000000000..09ef36faae --- /dev/null +++ b/documentation/overview-manual/svg/bitbake_tasks_map.svg @@ -0,0 +1,4 @@ + + + +
tmp
tmp
work
wo= rk
${MULTIMACH_TA= RGET_OS}
${MULTIMACH_TARGET_OS}
libhello
libhello
say= hello
sayhello<= switch>
0.1-r0
0.1-r0
0.1-r0
0.1-r0=
d= ownloads
downloads
git2
git2=
github.com.= <username>.sayhello
github.com.<username>.sayhello
= github.com.<username>.libhello
github.com.<username>.libhe= llo
git
git
Makefile
Makefile
LICENSE
LICENSE
fix.= patch
fix.patch
hellolib.c
hellolib.= c
<= div style=3D"display: inline-block; font-size: 12px; font-family: "Lib= eration Sans"; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: = all; white-space: normal; overflow-wrap: normal;">temp
temp
sysroot-des= tdir
sysroot-destdir
=
recipe-sysroo= t
recipe-sysroot
<= div style=3D"display: inline-block; font-size: 12px; font-family: "Lib= eration Sans"; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: = all; white-space: normal; overflow-wrap: normal;">image
image<= rect x=3D"3029" y=3D"2720" width=3D"174" height=3D"160" rx=3D"24" ry=3D"24"= fill=3D"rgb(255, 255, 255)" stroke=3D"rgb(0, 0, 0)" stroke-width=3D"4" poi= nter-events=3D"all"/>=
usr
usr<= /g>
include
include
lib
=
= lib
hellolib.h
hellolib.h
libhello.so.1
libhello.so.1
li= bhello.so.1.0
libhello.so.1.0
Everything in image fol= der that is present in SYSROOT_DIRS will be copied here.
=
Every= thing in image...= <= switch>
do_install
= do_install
libhello.so.1.0
libhello.so.1.0<= /g>
do_configure
do_configure
hellolib.h
he= llolib.h
<= div data-drawio-colors=3D"color: rgb(0, 0, 0); " style=3D"box-sizing: borde= r-box; font-size: 0px; text-align: center;">
do_patch
do_patch
do_unpack
do_unpack
6
=
6
<= b style=3D"font-size: 15px;">7
= 7
8
8
do_compile
do_compile
9
9
10
10
<= div style=3D"display: inline-block; font-size: 12px; font-family: "Lib= eration Sans"; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: = all; white-space: nowrap;">do_populate_sysroot
do_p= opulate_sysroot
11
11
gi= t
git
Makefile
Makefile<= /text>
LICENSE
<= text x=3D"1757" y=3D"724" fill=3D"#333333" font-family=3D"Liberation Sans" = font-size=3D"12px" text-anchor=3D"middle">LICENSE=
sayhello.c
sayhello.c=
sayhell= o
sayhello
image
image
usr
usr<= rect x=3D"7608.52" y=3D"3080" width=3D"215" height=3D"160" rx=3D"24" ry=3D"= 24" fill=3D"rgb(255, 255, 255)" stroke=3D"rgb(0, 0, 0)" stroke-width=3D"4" = pointer-events=3D"all"/>
bin
bin
libhello-0.1
libhello-0.1
S =3D "${WORKDIR}/git"
=
S =3D = "${WORKDIR}/git"<= foreignObject pointer-events=3D"none" width=3D"100%" height=3D"100%" requir= edFeatures=3D"http://www.w3.org/TR/SVG11/feature#Extensibility" style=3D"ov= erflow: visible; text-align: left;">
WORKDIR
WORKDIR
D
D<= switch>
S=
S
B=
B
S=
S
T=
T
libdir
libdir
includedir
includedir
RECIPE_SYSROOT
RE= CIPE_SYSROOT
S
S<= /switch>
B
B<= /switch>
D
D<= /switch>
WORKDIR
WOR= KDIR
do_prepare_recipe_sysroot
do_prepare_recipe_sysroot
sayhello
sayhello
usr
usr
include
include
lib
<= /div>lib
hellolib.h
hellolib.h
libhello.so.1
libhello.so.1
libh= ello.so.1.0
libhello.so.1.0<= g transform=3D"translate(-0.5 -0.5)scale(4)">
This also contains other files from other=C2=A0
= dependencies. Default dependencies are:
basically gcc= , compilerlibs and libc
This also contains other files from other.= ..
SYSROOT_DESTDIR
<= /div>
SYSROOT_DESTDIR=
Github
=
Github<= /text>
do_= fetch
do_fetch
<= font data-font-src=3D"https://fonts.googleapis.com/css?family=3DArchitects+= Daughter" color=3D"#ffffff" size=3D"1">1<= /font>
1
do_fetch
do_fetch
5
5
do_configure
= do_configur= e
12
12
do_compi= le
do_compile
13<= /b>
13
do_unpack
do_unpack
2
2
conf
co= nf=
local.conf
local.conf
bblayers.conf
bblayers.conf
S defaults generally to ${WORKDIR}/${BPN}-${PV}
In git recipes change it to ${WORKDIR}/git<= /font>
S defaults generally to ${WORKDIR}/${B...<= rect x=3D"6228" y=3D"2700" width=3D"440" height=3D"160" fill=3D"#eeeeee" st= roke=3D"#36393d" stroke-width=3D"4" pointer-events=3D"all"/>
do_patch
(No= patches)
do_patch...
3
3
4
4
do_install
do_install
14
14
pa= ckage
package
PKGD
PKGD
A copy of=C2=A0${D}
excludi= ng
/sysroot-only
A copy of=C2=A0${D}...
do_package
do_package
= 15
15<= switch>
packages-split
=
packages-split16
deploy-<= i>pkg
PKGDEST
PKGDEST
sayhello
s= ayhello
usr
usr
bin<= /font>
bin
Folders created here are present in = PACKAGES variable, BitBake knows what and where to put things us= ing the FILES variable, example: FILES:${PN} fi= les will go to ${PN}=C2=A0folder which is in PACKAGES
Folders created here are present in PACKAGES variable..= .
do_package
do_package
16
deploy-pkg
${PAC= KAGE_ARCH}
${PACKAGE_ARCH}<= foreignObject pointer-events=3D"none" width=3D"100%" height=3D"100%" requir= edFeatures=3D"http://www.w3.org/TR/SVG11/feature#Extensibility" style=3D"ov= erflow: visible; text-align: left;">This can be rpms, d= ebs or ipks....
sayhello-0.1-r0.${PACKAGE_ARCH}.pkg
sayhello-0.1-r0.= ${PACKAGE_ARCH}.pkg
This task also depends on PACKAGE_CLASSES,
pkg=C2=A0can be rpm, deb<= /u> or ipk for package_rpm,
pack= age_deb or package_ipk=C2=A0res= pectively.
The generated package generally named using:
${P= N}, ${PR}, ${PACKAGE_ARCH} and=C2=A0= pkg
This task also depends on PACKAGE_= CLASSES,...
deploy
deploy
=
${DEPLOY_DIR_pkg}
${DEPLOY_DIR_pkg= }
do_package_write_pkg
do_package_write_pkg <= /text>
17
17=
${PACKAGE_ARCH}
${PACKAGE_ARCH}=
For packages, this can be I= PK, RPM or DEB (check step 17)
For packages, this can be IPK, RPM or DEB (check step 17)=
DEPLOY_DIR
<= /foreignObject>DEPLOY_DIR
TMPDIR
TMPDIR
DL_DIR
DL_DIR
<= div data-drawio-colors=3D"color: rgb(0, 0, 0); " style=3D"box-sizing: borde= r-box; font-size: 0px; text-align: center;">
do_package_write_pkg
do_package_write_pkg=
18
18=
This can be PKGWRITEDIR= RPM, PKGWRITEDIRDEB or PKGWRITEDIRIPK f= or package_rpm, package_deb
or packa= ge_ipk respectively
<= text x=3D"2615" y=3D"244" fill=3D"rgb(0, 0, 0)" font-family=3D"Liberation S= ans" font-size=3D"12px" text-anchor=3D"middle">This can be PKGWRITEDIRRPM, = PKGWRITEDIRDEB or PKGWRITEDIRIPK for pack...
License checking happens in do_populate_lic after do_patch
and before th= at a checksum check
happends on=C2=A0LIC_FILES_CHKSUM if= the
license is not CLOSED
License che= cking happens in do_populate_lic after do_pa...
This variable is us= ed to separate recipes
based on their target. This has value of
<= b>${PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}
This variable is used to separate recipes...=
conf-notes.txt
conf-notes.txt
sstate-cache
sstate-cache
SSTATE_DIR
SSTATE_DIR
sayhello
sayhello=
This = folder contains cache for recipes build output, this is used by BitBake, if= the recipe checksum did not change it knows that the output to use is the = same.
This folder contains cache for recipes build output= , this is used by...
These directories can be shared accross builds to = save disk space and build time
These directories ca= n be shared accross builds to save disk space an...<= switch>
This = directory contains other output directories such as images, <= b>sdk and licenses
<= /foreignObject>This directo= ry contains other output directories such as images, sdk...=
This file cont= ains all layers that BitBake should consider when looking for metada= ta.
This file contains all layer...<= rect x=3D"5396" y=3D"20" width=3D"1760" height=3D"160" fill=3D"none" stroke= =3D"rgb(0, 0, 0)" stroke-width=3D"4" stroke-dasharray=3D"12 12" pointer-eve= nts=3D"all"/>
=
This is base configur= ation file containing essential user config such as MACHINE a= nd DISTRO
This is base configuration file c= ontaining essential user config such as...
The message to show after = source oe-init-build-env
= The message to show after s= ource oe-init...Text is not SVG - cannot display= \ No newline at end of file --=20 2.34.1
This can be <= b>rpms, debs or ipks.
These are= provided by
package_rpm, package_deb and = package_ipk classes respectively, use PACKAGE_CLASSES for that as
content of PACKAGE_CLASSES will be ap= pended
to INHERIT