* [RFC][PATCH] bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link @ 2018-09-27 9:11 Martin Jansa 2018-09-27 9:34 ` ✗ patchtest: failure for " Patchwork 2018-09-27 11:15 ` [RFC][PATCH] " Richard Purdie 0 siblings, 2 replies; 4+ messages in thread From: Martin Jansa @ 2018-09-27 9:11 UTC (permalink / raw) To: openembedded-core * just RFC, the part for images isn't finished yet and there is still some issue with DATETIME when kernel artifacts are used from sstate, this is just to validate the idea behind [YOCTO #12937] before finishing the implementation (it's already finished and used by various LGE builds, but having simpler way of doing it directly in oe-core mighe be useful for others). * move IMAGE_VERSION_SUFFIX from _NAME variables to _LINK_NAME that way e.g. kernel.do_deploy can be reused from sstate to provide "version-less" artifacts and then very fast do_deploy_links task just adds links with consistent suffixes * create hard links instead of symlinks, so that whatever version the filename says is really there * some IMAGE_FSTYPES might need the "version-less" IMAGE_NAME file to be removed first or they might either append or update the content of the image instead of creating new image file from scratch - I have seen this only with one proprietary format we generate with our own tool, so hopefully this isn't very common * this is basically the mechanism are using in webOS with WEBOS_IMAGE_NAME_SUFFIX which is for official builds set from jenkins job and then all artifacts (images as well as corresponding kernel files) have the same version string, the implementation "from outside" is a bit tricky as shown in webOS OSE, the variable modifications: https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/conf/distro/include/webos.inc#L65 and then various bbclasses to hook do_webos_deploy_fixup task creating the hardlinks for possible artifacts: https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/webos_deploy.bbclass https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/kernel.bbclass https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/image.bbclass so hopefully with all these changes in oe-core other project can achieve the same just by setting one variable IMAGE_VERSION_SUFFIX [YOCTO #12937] Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> --- meta/classes/kernel-artifact-names.bbclass | 4 +-- meta/classes/kernel.bbclass | 40 ++++++++++++++++------ meta/conf/bitbake.conf | 4 +-- 3 files changed, 33 insertions(+), 15 deletions(-) diff --git a/meta/classes/kernel-artifact-names.bbclass b/meta/classes/kernel-artifact-names.bbclass index bbeecba7bd..84ec193b5a 100644 --- a/meta/classes/kernel-artifact-names.bbclass +++ b/meta/classes/kernel-artifact-names.bbclass @@ -1,5 +1,5 @@ -KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}" -KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}" +KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}" +KERNEL_ARTIFACT_LINK_NAME ?= "${KERNEL_ARTIFACT_NAME}${IMAGE_VERSION_SUFFIX}" KERNEL_IMAGE_NAME ?= "${KERNEL_ARTIFACT_NAME}" KERNEL_IMAGE_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}" diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass index d0fbbd1989..7aaebb56b4 100644 --- a/meta/classes/kernel.bbclass +++ b/meta/classes/kernel.bbclass @@ -667,25 +667,18 @@ kernel_do_deploy() { fi for imageType in ${KERNEL_IMAGETYPES} ; do - base_name=${imageType}-${KERNEL_IMAGE_NAME} - install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType} $deployDir/${base_name}.bin - symlink_name=${imageType}-${KERNEL_IMAGE_LINK_NAME} - ln -sf ${base_name}.bin $deployDir/${symlink_name}.bin - ln -sf ${base_name}.bin $deployDir/${imageType} + install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType} $deployDir/${imageType}-${KERNEL_IMAGE_NAME}.bin + ln -sf ${imageType}-${KERNEL_IMAGE_NAME}.bin $deployDir/${imageType} done if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e '^CONFIG_MODULES=y$' .config); then mkdir -p ${D}${root_prefix}/lib tar -cvzf $deployDir/modules-${MODULE_TARBALL_NAME}.tgz -C ${D}${root_prefix} lib - ln -sf modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz fi if [ ! -z "${INITRAMFS_IMAGE}" -a x"${INITRAMFS_IMAGE_BUNDLE}" = x1 ]; then for imageType in ${KERNEL_IMAGETYPES} ; do - initramfs_base_name=${imageType}-${INITRAMFS_NAME} - initramfs_symlink_name=${imageType}-${INITRAMFS_LINK_NAME} - install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType}.initramfs $deployDir/${initramfs_base_name}.bin - ln -sf ${initramfs_base_name}.bin $deployDir/${initramfs_symlink_name}.bin + install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType}.initramfs $deployDir/${imageType}-${INITRAMFS_NAME}.bin done fi } @@ -695,7 +688,32 @@ do_deploy[prefuncs] += "package_get_auto_pr" addtask deploy after do_populate_sysroot do_packagedata -EXPORT_FUNCTIONS do_deploy +kernel_do_deploy_links() { + deployDir="${DEPLOY_DIR_IMAGE}" + if [ -n "${KERNEL_DEPLOYSUBDIR}" ]; then + deployDir="${DEPLOY_DIR_IMAGE}/${KERNEL_DEPLOYSUBDIR}" + mkdir "$deployDir" + fi + + for imageType in ${KERNEL_IMAGETYPES} ; do + ln -vf $deployDir/${imageType}-${KERNEL_IMAGE_NAME}.bin $deployDir/${imageType}-${KERNEL_IMAGE_LINK_NAME}.bin + done + + if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e '^CONFIG_MODULES=y$' .config); then + ln -vf $deployDir/modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz + fi + + if [ ! -z "${INITRAMFS_IMAGE}" -a "${INITRAMFS_IMAGE_BUNDLE}" = "1" ]; then + for imageType in ${KERNEL_IMAGETYPES} ; do + ln -vf ${imageType}-${INITRAMFS_NAME}.bin ${imageType}-${INITRAMFS_LINK_NAME}.bin + done + fi +} +do_deploy_links[prefuncs] += "package_get_auto_pr" + +addtask deploy_links after do_deploy before do_build + +EXPORT_FUNCTIONS do_deploy do_deploy_links # Add using Device Tree support inherit kernel-devicetree diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 93aee1ae46..017cb163ba 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -447,8 +447,8 @@ IMAGE_ROOTFS = "${WORKDIR}/rootfs" IMAGE_BASENAME = "${PN}" IMAGE_VERSION_SUFFIX = "-${DATETIME}" IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME" -IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}" -IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}" +IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}" +IMAGE_LINK_NAME = "${IMAGE_NAME}${IMAGE_VERSION_SUFFIX}" # This option allows for a percentage overage of the actual image size rather than a # fixed extra space, this is space needed for initial startup and basic operations. -- 2.17.1 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* ✗ patchtest: failure for bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link 2018-09-27 9:11 [RFC][PATCH] bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link Martin Jansa @ 2018-09-27 9:34 ` Patchwork 2018-09-27 11:15 ` [RFC][PATCH] " Richard Purdie 1 sibling, 0 replies; 4+ messages in thread From: Patchwork @ 2018-09-27 9:34 UTC (permalink / raw) To: Martin Jansa; +Cc: openembedded-core == Series Details == Series: bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link Revision: 1 URL : https://patchwork.openembedded.org/series/14258/ State : failure == Summary == Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures: * Patch [RFC] bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link Issue Commit shortlog is too long [test_shortlog_length] Suggested fix Edit shortlog so that it is 90 characters or less (currently 119 characters) If you believe any of these test results are incorrect, please reply to the mailing list (openembedded-core@lists.openembedded.org) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...). --- Guidelines: https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [RFC][PATCH] bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link 2018-09-27 9:11 [RFC][PATCH] bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link Martin Jansa 2018-09-27 9:34 ` ✗ patchtest: failure for " Patchwork @ 2018-09-27 11:15 ` Richard Purdie 2018-09-27 12:13 ` Martin Jansa 1 sibling, 1 reply; 4+ messages in thread From: Richard Purdie @ 2018-09-27 11:15 UTC (permalink / raw) To: Martin Jansa, openembedded-core Hi Martin, In the commit message you say a lot about what you've changed but not so much about why the changes are important and the advantages they bring. There are tradeoffs, for example symlinks make it clear which artefact they're really pointing at, hardlinks hide that fact, you need to go and look at inode numbers to figure out which of several artefacts one might be pointing at. I'm not sure that is an improvement. I'm also slightly concerned that we need to bypass sstate control, the whole intent there was to ensure that output is reproducible and consistently restored. Adding in a new task means do_build target usage will work but any of the code that has do_deploy as a dependency (e.g. an recrdep) will now also need to consider do_deploy_links. For that reason alone, I'm not sure we can do this. Cheers, Richard On Thu, 2018-09-27 at 09:11 +0000, Martin Jansa wrote: > * just RFC, the part for images isn't finished yet and there is > still some issue with DATETIME when kernel artifacts are used > from sstate, this is just to validate the idea behind > [YOCTO #12937] before finishing the implementation (it's already > finished and used by various LGE builds, but having simpler > way of doing it directly in oe-core mighe be useful for others). > > * move IMAGE_VERSION_SUFFIX from _NAME variables to _LINK_NAME > that way e.g. kernel.do_deploy can be reused from sstate to > provide "version-less" artifacts and then very fast > do_deploy_links task just adds links with consistent suffixes > * create hard links instead of symlinks, so that whatever version > the filename says is really there > * some IMAGE_FSTYPES might need the "version-less" IMAGE_NAME file > to be removed first or they might either append or update the > content of the image instead of creating new image file from > scratch - I have seen this only with one proprietary format we > generate with our own tool, so hopefully this isn't very common > * this is basically the mechanism are using in webOS with > WEBOS_IMAGE_NAME_SUFFIX which is for official builds set from > jenkins job and then all artifacts (images as well as corresponding > kernel files) have the same version string, the implementation > "from outside" is a bit tricky as shown in webOS OSE, the variable > modifications: > https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e > 44a132d01297ff478248/meta-webos/conf/distro/include/webos.inc#L65 > and then various bbclasses to hook do_webos_deploy_fixup task > creating > the hardlinks for possible artifacts: > https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e > 44a132d01297ff478248/meta-webos/classes/webos_deploy.bbclass > https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e > 44a132d01297ff478248/meta-webos/classes/kernel.bbclass > https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e > 44a132d01297ff478248/meta-webos/classes/image.bbclass > so hopefully with all these changes in oe-core other project can > achieve the same just by setting one variable IMAGE_VERSION_SUFFIX > > [YOCTO #12937] > > Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> > --- > meta/classes/kernel-artifact-names.bbclass | 4 +-- > meta/classes/kernel.bbclass | 40 ++++++++++++++++-- > ---- > meta/conf/bitbake.conf | 4 +-- > 3 files changed, 33 insertions(+), 15 deletions(-) > > diff --git a/meta/classes/kernel-artifact-names.bbclass > b/meta/classes/kernel-artifact-names.bbclass > index bbeecba7bd..84ec193b5a 100644 > --- a/meta/classes/kernel-artifact-names.bbclass > +++ b/meta/classes/kernel-artifact-names.bbclass > @@ -1,5 +1,5 @@ > -KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}- > ${MACHINE}${IMAGE_VERSION_SUFFIX}" > -KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}" > +KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}" > +KERNEL_ARTIFACT_LINK_NAME ?= > "${KERNEL_ARTIFACT_NAME}${IMAGE_VERSION_SUFFIX}" > > KERNEL_IMAGE_NAME ?= "${KERNEL_ARTIFACT_NAME}" > KERNEL_IMAGE_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}" > diff --git a/meta/classes/kernel.bbclass > b/meta/classes/kernel.bbclass > index d0fbbd1989..7aaebb56b4 100644 > --- a/meta/classes/kernel.bbclass > +++ b/meta/classes/kernel.bbclass > @@ -667,25 +667,18 @@ kernel_do_deploy() { > fi > > for imageType in ${KERNEL_IMAGETYPES} ; do > - base_name=${imageType}-${KERNEL_IMAGE_NAME} > - install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType} > $deployDir/${base_name}.bin > - symlink_name=${imageType}-${KERNEL_IMAGE_LINK_NAME} > - ln -sf ${base_name}.bin > $deployDir/${symlink_name}.bin > - ln -sf ${base_name}.bin $deployDir/${imageType} > + install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType} > $deployDir/${imageType}-${KERNEL_IMAGE_NAME}.bin > + ln -sf ${imageType}-${KERNEL_IMAGE_NAME}.bin > $deployDir/${imageType} > done > > if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e > '^CONFIG_MODULES=y$' .config); then > mkdir -p ${D}${root_prefix}/lib > tar -cvzf $deployDir/modules- > ${MODULE_TARBALL_NAME}.tgz -C ${D}${root_prefix} lib > - ln -sf modules-${MODULE_TARBALL_NAME}.tgz > $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz > fi > > if [ ! -z "${INITRAMFS_IMAGE}" -a > x"${INITRAMFS_IMAGE_BUNDLE}" = x1 ]; then > for imageType in ${KERNEL_IMAGETYPES} ; do > - initramfs_base_name=${imageType}- > ${INITRAMFS_NAME} > - initramfs_symlink_name=${imageType}- > ${INITRAMFS_LINK_NAME} > - install -m 0644 > ${KERNEL_OUTPUT_DIR}/${imageType}.initramfs > $deployDir/${initramfs_base_name}.bin > - ln -sf ${initramfs_base_name}.bin > $deployDir/${initramfs_symlink_name}.bin > + install -m 0644 > ${KERNEL_OUTPUT_DIR}/${imageType}.initramfs $deployDir/${imageType}- > ${INITRAMFS_NAME}.bin > done > fi > } > @@ -695,7 +688,32 @@ do_deploy[prefuncs] += "package_get_auto_pr" > > addtask deploy after do_populate_sysroot do_packagedata > > -EXPORT_FUNCTIONS do_deploy > +kernel_do_deploy_links() { > + deployDir="${DEPLOY_DIR_IMAGE}" > + if [ -n "${KERNEL_DEPLOYSUBDIR}" ]; then > + deployDir="${DEPLOY_DIR_IMAGE}/${KERNEL_DEPLOYSUBDIR > }" > + mkdir "$deployDir" > + fi > + > + for imageType in ${KERNEL_IMAGETYPES} ; do > + ln -vf $deployDir/${imageType}- > ${KERNEL_IMAGE_NAME}.bin $deployDir/${imageType}- > ${KERNEL_IMAGE_LINK_NAME}.bin > + done > + > + if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e > '^CONFIG_MODULES=y$' .config); then > + ln -vf $deployDir/modules-${MODULE_TARBALL_NAME}.tgz > $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz > + fi > + > + if [ ! -z "${INITRAMFS_IMAGE}" -a > "${INITRAMFS_IMAGE_BUNDLE}" = "1" ]; then > + for imageType in ${KERNEL_IMAGETYPES} ; do > + ln -vf ${imageType}-${INITRAMFS_NAME}.bin > ${imageType}-${INITRAMFS_LINK_NAME}.bin > + done > + fi > +} > +do_deploy_links[prefuncs] += "package_get_auto_pr" > + > +addtask deploy_links after do_deploy before do_build > + > +EXPORT_FUNCTIONS do_deploy do_deploy_links > > # Add using Device Tree support > inherit kernel-devicetree > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf > index 93aee1ae46..017cb163ba 100644 > --- a/meta/conf/bitbake.conf > +++ b/meta/conf/bitbake.conf > @@ -447,8 +447,8 @@ IMAGE_ROOTFS = "${WORKDIR}/rootfs" > IMAGE_BASENAME = "${PN}" > IMAGE_VERSION_SUFFIX = "-${DATETIME}" > IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME" > -IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}" > -IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}" > +IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}" > +IMAGE_LINK_NAME = "${IMAGE_NAME}${IMAGE_VERSION_SUFFIX}" > > # This option allows for a percentage overage of the actual image > size rather than a > # fixed extra space, this is space needed for initial startup and > basic operations. > -- > 2.17.1 > ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [RFC][PATCH] bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link 2018-09-27 11:15 ` [RFC][PATCH] " Richard Purdie @ 2018-09-27 12:13 ` Martin Jansa 0 siblings, 0 replies; 4+ messages in thread From: Martin Jansa @ 2018-09-27 12:13 UTC (permalink / raw) To: Richard Purdie; +Cc: Patches and discussions about the oe-core layer [-- Attachment #1: Type: text/plain, Size: 11808 bytes --] The reason is that for our builds we set the IMAGE_VERSION_SUFFIX based on jenkins job number which creates them (or you can imagine IMAGE_VERSION_SUFFIX being set to something like "release-2.6"). With current metadata we have 2 options: 1) keep IMAGE_VERSION_SUFFIX as is, but don't vardepexclude the build number (DATETIME in the default value) But this causes e.g. kernel.do_deploy signature to be changed and whole do_deploy task re-executed (instead of reused from sstate if it's the same except the output files names), which in turn causes whole kernel to be rebuilt in every build, because to run do_deploy we need do_install etc (and these builds are usually using rm_work and TMPDIR is empty before each job). 2) leave IMAGE_VERSION_SUFFIX as is, but rename the artifacts after the build When the image artifact says "image-release-2.6.img" and kernel image from the same build says "uImage-release-2.5.bin" (because kernel wasn't modified at all since previous release), people will be confused. This is what we were seeing in CI which was usually sending just the image, but then for some new MACHINEs we needed to send kernel artifact separately and sometimes the artifact wasn't found with expected name. Then we changed the build number (aka IMAGE_VERSION_SUFFIX) to be included in do_deploy signatures which lead to 1) and kernel rebuilds every time To solve both, we've added webos_deploy_fixup tasks (in many places) which just adds the hardlinks with expected filenames (so that other systems just using the artifacts doesn't need to know anything about internal issues with sstate reuse, nor needing external script to rename things after the build). Hardlinks are better than symlinks in this case, because you don't want to have symlink image-release-2.5.img -> image.img still in the deploy directory after you do another build which creates new image.img and new symlink image-release-2.6.img -> image.img If someone downloads image-release-2.5.img "too late" he will get 2.6 version instead. Hardlinks don't have this issue and yes the need to check the inode numbers might be confusing to some, but when do you really care where does it point? In our use-case we always care only about the *"release-2.5"* and the artifacts without the version string are considered as intermediate files created by the build or "whatever version was built last in this build directory", pretty much the same as version-less symlinks in current build (which are easy to resolve when looking at the filesystem, but exposing those symlinks for download e.g. over HTTP hides where it points completely. I'm not sure I understand your concern about bypassing sstate control, we still want to reuse kernel.do_deploy and image.do_image_complete. Just with additional very fast intentionally not-covered-by-sstate task before do_build to create consistent names. On Thu, Sep 27, 2018 at 1:15 PM Richard Purdie < richard.purdie@linuxfoundation.org> wrote: > Hi Martin, > > In the commit message you say a lot about what you've changed but not > so much about why the changes are important and the advantages they > bring. > > There are tradeoffs, for example symlinks make it clear which artefact > they're really pointing at, hardlinks hide that fact, you need to go > and look at inode numbers to figure out which of several artefacts one > might be pointing at. I'm not sure that is an improvement. > > I'm also slightly concerned that we need to bypass sstate control, the > whole intent there was to ensure that output is reproducible and > consistently restored. Adding in a new task means do_build target usage > will work but any of the code that has do_deploy as a dependency (e.g. > an recrdep) will now also need to consider do_deploy_links. For that > reason alone, I'm not sure we can do this. > > Cheers, > > Richard > > On Thu, 2018-09-27 at 09:11 +0000, Martin Jansa wrote: > > * just RFC, the part for images isn't finished yet and there is > > still some issue with DATETIME when kernel artifacts are used > > from sstate, this is just to validate the idea behind > > [YOCTO #12937] before finishing the implementation (it's already > > finished and used by various LGE builds, but having simpler > > way of doing it directly in oe-core mighe be useful for others). > > > > * move IMAGE_VERSION_SUFFIX from _NAME variables to _LINK_NAME > > that way e.g. kernel.do_deploy can be reused from sstate to > > provide "version-less" artifacts and then very fast > > do_deploy_links task just adds links with consistent suffixes > > * create hard links instead of symlinks, so that whatever version > > the filename says is really there > > * some IMAGE_FSTYPES might need the "version-less" IMAGE_NAME file > > to be removed first or they might either append or update the > > content of the image instead of creating new image file from > > scratch - I have seen this only with one proprietary format we > > generate with our own tool, so hopefully this isn't very common > > * this is basically the mechanism are using in webOS with > > WEBOS_IMAGE_NAME_SUFFIX which is for official builds set from > > jenkins job and then all artifacts (images as well as corresponding > > kernel files) have the same version string, the implementation > > "from outside" is a bit tricky as shown in webOS OSE, the variable > > modifications: > > https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e > > 44a132d01297ff478248/meta-webos/conf/distro/include/webos.inc#L65 > > and then various bbclasses to hook do_webos_deploy_fixup task > > creating > > the hardlinks for possible artifacts: > > https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e > > 44a132d01297ff478248/meta-webos/classes/webos_deploy.bbclass > > https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e > > 44a132d01297ff478248/meta-webos/classes/kernel.bbclass > > https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e > > 44a132d01297ff478248/meta-webos/classes/image.bbclass > > so hopefully with all these changes in oe-core other project can > > achieve the same just by setting one variable IMAGE_VERSION_SUFFIX > > > > [YOCTO #12937] > > > > Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> > > --- > > meta/classes/kernel-artifact-names.bbclass | 4 +-- > > meta/classes/kernel.bbclass | 40 ++++++++++++++++-- > > ---- > > meta/conf/bitbake.conf | 4 +-- > > 3 files changed, 33 insertions(+), 15 deletions(-) > > > > diff --git a/meta/classes/kernel-artifact-names.bbclass > > b/meta/classes/kernel-artifact-names.bbclass > > index bbeecba7bd..84ec193b5a 100644 > > --- a/meta/classes/kernel-artifact-names.bbclass > > +++ b/meta/classes/kernel-artifact-names.bbclass > > @@ -1,5 +1,5 @@ > > -KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}- > > ${MACHINE}${IMAGE_VERSION_SUFFIX}" > > -KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}" > > +KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}" > > +KERNEL_ARTIFACT_LINK_NAME ?= > > "${KERNEL_ARTIFACT_NAME}${IMAGE_VERSION_SUFFIX}" > > > > KERNEL_IMAGE_NAME ?= "${KERNEL_ARTIFACT_NAME}" > > KERNEL_IMAGE_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}" > > diff --git a/meta/classes/kernel.bbclass > > b/meta/classes/kernel.bbclass > > index d0fbbd1989..7aaebb56b4 100644 > > --- a/meta/classes/kernel.bbclass > > +++ b/meta/classes/kernel.bbclass > > @@ -667,25 +667,18 @@ kernel_do_deploy() { > > fi > > > > for imageType in ${KERNEL_IMAGETYPES} ; do > > - base_name=${imageType}-${KERNEL_IMAGE_NAME} > > - install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType} > > $deployDir/${base_name}.bin > > - symlink_name=${imageType}-${KERNEL_IMAGE_LINK_NAME} > > - ln -sf ${base_name}.bin > > $deployDir/${symlink_name}.bin > > - ln -sf ${base_name}.bin $deployDir/${imageType} > > + install -m 0644 ${KERNEL_OUTPUT_DIR}/${imageType} > > $deployDir/${imageType}-${KERNEL_IMAGE_NAME}.bin > > + ln -sf ${imageType}-${KERNEL_IMAGE_NAME}.bin > > $deployDir/${imageType} > > done > > > > if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e > > '^CONFIG_MODULES=y$' .config); then > > mkdir -p ${D}${root_prefix}/lib > > tar -cvzf $deployDir/modules- > > ${MODULE_TARBALL_NAME}.tgz -C ${D}${root_prefix} lib > > - ln -sf modules-${MODULE_TARBALL_NAME}.tgz > > $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz > > fi > > > > if [ ! -z "${INITRAMFS_IMAGE}" -a > > x"${INITRAMFS_IMAGE_BUNDLE}" = x1 ]; then > > for imageType in ${KERNEL_IMAGETYPES} ; do > > - initramfs_base_name=${imageType}- > > ${INITRAMFS_NAME} > > - initramfs_symlink_name=${imageType}- > > ${INITRAMFS_LINK_NAME} > > - install -m 0644 > > ${KERNEL_OUTPUT_DIR}/${imageType}.initramfs > > $deployDir/${initramfs_base_name}.bin > > - ln -sf ${initramfs_base_name}.bin > > $deployDir/${initramfs_symlink_name}.bin > > + install -m 0644 > > ${KERNEL_OUTPUT_DIR}/${imageType}.initramfs $deployDir/${imageType}- > > ${INITRAMFS_NAME}.bin > > done > > fi > > } > > @@ -695,7 +688,32 @@ do_deploy[prefuncs] += "package_get_auto_pr" > > > > addtask deploy after do_populate_sysroot do_packagedata > > > > -EXPORT_FUNCTIONS do_deploy > > +kernel_do_deploy_links() { > > + deployDir="${DEPLOY_DIR_IMAGE}" > > + if [ -n "${KERNEL_DEPLOYSUBDIR}" ]; then > > + deployDir="${DEPLOY_DIR_IMAGE}/${KERNEL_DEPLOYSUBDIR > > }" > > + mkdir "$deployDir" > > + fi > > + > > + for imageType in ${KERNEL_IMAGETYPES} ; do > > + ln -vf $deployDir/${imageType}- > > ${KERNEL_IMAGE_NAME}.bin $deployDir/${imageType}- > > ${KERNEL_IMAGE_LINK_NAME}.bin > > + done > > + > > + if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e > > '^CONFIG_MODULES=y$' .config); then > > + ln -vf $deployDir/modules-${MODULE_TARBALL_NAME}.tgz > > $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz > > + fi > > + > > + if [ ! -z "${INITRAMFS_IMAGE}" -a > > "${INITRAMFS_IMAGE_BUNDLE}" = "1" ]; then > > + for imageType in ${KERNEL_IMAGETYPES} ; do > > + ln -vf ${imageType}-${INITRAMFS_NAME}.bin > > ${imageType}-${INITRAMFS_LINK_NAME}.bin > > + done > > + fi > > +} > > +do_deploy_links[prefuncs] += "package_get_auto_pr" > > + > > +addtask deploy_links after do_deploy before do_build > > + > > +EXPORT_FUNCTIONS do_deploy do_deploy_links > > > > # Add using Device Tree support > > inherit kernel-devicetree > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf > > index 93aee1ae46..017cb163ba 100644 > > --- a/meta/conf/bitbake.conf > > +++ b/meta/conf/bitbake.conf > > @@ -447,8 +447,8 @@ IMAGE_ROOTFS = "${WORKDIR}/rootfs" > > IMAGE_BASENAME = "${PN}" > > IMAGE_VERSION_SUFFIX = "-${DATETIME}" > > IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME" > > -IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}" > > -IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}" > > +IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}" > > +IMAGE_LINK_NAME = "${IMAGE_NAME}${IMAGE_VERSION_SUFFIX}" > > > > # This option allows for a percentage overage of the actual image > > size rather than a > > # fixed extra space, this is space needed for initial startup and > > basic operations. > > -- > > 2.17.1 > > > [-- Attachment #2: Type: text/html, Size: 14625 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2018-09-27 12:13 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2018-09-27 9:11 [RFC][PATCH] bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link Martin Jansa 2018-09-27 9:34 ` ✗ patchtest: failure for " Patchwork 2018-09-27 11:15 ` [RFC][PATCH] " Richard Purdie 2018-09-27 12:13 ` Martin Jansa
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox